Skip to content

Introduce coffea-style systematics

Massimiliano Galli requested to merge magalli/HiggsDNA:start-systematics into master

This PR introduces the coffea-style systematics in the Hgg base workflow. As an example, a dummy systematic PhotonPtScale is introduced. Major aspect to note is that in such a case the output of this kind of workflow has the following structure:

.
├── nominal
│   ├── 1c2a00ae-7aac-11ec-97b1-29e75d9fbeef_%2FEvents%3B1_0-78349.parquet
│   ├── 292a7d56-7aac-11ec-859a-70e75d9fbeef_%2FEvents%3B1_0-78170.parquet
│   └── a0e5db9c-7aac-11ec-8df4-020f010abeef_%2FEvents%3B1_0-9665.parquet
├── PhotonPtScale_down
│   ├── 1c2a00ae-7aac-11ec-97b1-29e75d9fbeef_%2FEvents%3B1_0-78349.parquet
│   ├── 292a7d56-7aac-11ec-859a-70e75d9fbeef_%2FEvents%3B1_0-78170.parquet
│   └── a0e5db9c-7aac-11ec-8df4-020f010abeef_%2FEvents%3B1_0-9665.parquet
└── PhotonPtScale_up
    ├── 1c2a00ae-7aac-11ec-97b1-29e75d9fbeef_%2FEvents%3B1_0-78349.parquet
    ├── 292a7d56-7aac-11ec-859a-70e75d9fbeef_%2FEvents%3B1_0-78170.parquet
    └── a0e5db9c-7aac-11ec-8df4-020f010abeef_%2FEvents%3B1_0-9665.parquet

i.e. one directory for each systematic variation, with the unaffected variables that are repeated through the different parquet files. I tried to implement a solution that followed the suggestion pointed out by @lgray here, but I found one major problem: apart from events passing or not passing a certain selection when a quantity is varied (which can indeed be easily adapted using null data entries) there are quantities, such as corrected shower shapes and isolations, whose values depend on the varied pt, and hence their value would have to be repeated anyways across the different systematic changes (i.e. different values for each nominal, PhotonPtScale_down and PhotonPtScale_up). Since I don't know if these cases are limited to shower shapes and isolations only, I thought that distinguishing a priori between the two kinds of variables (i.e. the ones that change depending on the changed quantity and the ones that do not) would have made the workflow messy, so I sticked to the original solution after a chat with @spigazzi. However, I'm more than willing to change it following suitable suggestions.

@lgray @smay can you please take a look and share opinions? thanks!

Edited by Massimiliano Galli

Merge request reports