Uncertainty digest for EGM
CAT has recently rolled out the uncertainty digest pages.
The aim of these pages is to provide an entry point, common to all POGs, for analyzers to navigate the uncertainty sources recommended by each POG, know where to pick them, ensure to apply all of them and know which name to give them in a Combine datacard.
The uncertainty digest is built based on the work by @kcormier, summarized in this presentation.
These uncertainty digest pages, crucially, include two concepts that are not part of the correctionlib JSON schema:
- The concept of
uncertainty_set, i.e. the set of uncertainties that should be applied together to achieve a complete set (e.g. JES reduced vs full splitting are two different sets, or BTV uncertainties for loose of medium WPs are two different sets). - The
combine_name, i.e. the name that needs to be applied to the corresponding parameter in the context of a Combine datacard. Important: mind that thecombine_nameencodes the correlation scheme, samecombine_namefor two uncertainties means the corresponding variations are controlled by a single nuisance parameter in the fit. One important aspect of these correlation schemes, that is not immediately relevant for a single analysis using a singleuncertainty_set, is the correlation of uncertainties betweenuncertainty_sets (e.g. for the Loose and Medium working point uncertainty sets of the same tagger).
The uncertainty_sets and the correlation schemes encoded in the combine_names have been derived from the twiki pages of the various POGs. The correlation of uncertainties between uncertainty_sets are somewhat special, as they are not addressed in depth in the twiki pages, and have been derived from the experience of existing combination, discussion with some of the experts and common sense.
It is important to notice that the correction digest does not provide new information, but presents in a coherent form what is in the jsonPOG-integration repository, and what is in the twiki pages. Some ambiguity is left in the twiki pages, especially regarding how to correlate uncertainties between uncertainty_sets. What is currently in the uncertainty digest is, and is intended to be, a good starting point from which analysis specific variations can be applied if deemed necessary.
It is also important to notice that the concepts of combine_name and uncertainty set, while not being part of the correctionlib JSON schema, are coded in a logic that resides in this repository. Should you update your recommendations, it will be important that you communicate it to CAT so that the proper actions can be taken to reflect the new recommendations.
It would probably be useful in the future if these concepts become part of the correctionlib JSON schema and are then completely controlled by the POGs.
We seek your confirmation that the interpretation we have given of your recommendation reflects how they were intended.
The uncertainty digest pages in the CAT docs referenced to above are extracted from the .md files contained in this repository, namely:
The overviews are rendered from this file for uncertainty_sets and this file for combine_names.
The tables further down the page are rendered from the tables listed here.
Please go through them at your earliest convenience and provide us with your feedback.