Automatic shape systematics?
I think this is the only big thing missing (other than fixing some bugs) before we can declare a first fully-usable version...
A recap of what plotIt expects: if there is a myH
histogram, and bla
systematic (in systematics
under the plotIt
section of the yml file), there should - for samples of type 'mc' - also be myH__blaup
and myH__bladown
histograms (or a separate file with such histograms, but that doesn't seem practical). If any of them are absent, the nominal histogram is used (i.e. effectively no systematic uncertainty from this source).
So we need a convenient way to fill such histograms alongside the nominal ones.
What is currently implemented, is that the HistogramsModule
has a systVars
list, and definePlots
is called with each value of that list (where "nominal" defines the list of nominal plots, and all others just get the histogram saved).
This is reasonably elegant (it is guaranteed to be correct, as long as all plots and affected selections carry unique names for all variations), but not the most efficient solution (it redefines - and recalculates - a number of things that could be reused).
All in all, there are essentially two kinds of shape systematics: different weights (all lepton, and b-tagging, systematics) and kinematic variations (rederiving quantities based on a collection with modified kinematics - essentially: jets).
The former only need a redefinition of the weight (which can be done without creating a new selection), whereas the latter do need new Filter
nodes, but should take advantage of the same jet variations calculator (which should be called only once per event).
One thing to try would be to modify the weight of an existing selection (find/replace?), to avoid the redefinitions "above" the where the jet variations calculator currently is.
Another option is to make the Selection
keep track of multiple weight variations, and make a Selection.weight(systVar="nominal")
method that gets the correct one, or even make the selection aware of which systematic variations are associated to it, and automatically make all the plots when requested.
The same could be done with collections; the trickiest part probably is the interplay between collections used in a selection (cut or weight) and in the definition of a variable to plot.
On the plus side, such an "automatic" option would allow to activate each systematic source in exactly one place - the one where it enters the analysis - so the system ensures only consistency, but the user stays in control of all the choices that matter (there could be an argument to the plot construction method to turn on systematics on a per-plot basis).
Plan of attack:
-
Selection
can tell which systematic variations affect it through weights -
scalefactors can change variation (nominal -> up/down) -
automatic varied histograms for weight-only systematics when constructing/initializing the Selection
andPlot
-
there is a way containers (and derived expressions) can tell which variations affect them -
Selection
can tell which container variations affect them -
Plot.make1D
/Plot.__init__
can tell which systematic variations affect it (also from plotvar and selection) -
arbitrary expressions (cut/plotvar) can change input containers (and scalefactors) -
Selection
makes nodes for modified containers -
Plot
construction makes all histograms -
list of shape systematics in yml is updated for plotItnot done to avoid surprises (and allow the user to select which to display), a list of all systematics will be printed