Skip to content

New trigger counts reference comparison tools and format

Rafal Bielski requested to merge rbielski/athena:new-ref-comp into master

New format for trigger counts summary was added in !37918 (merged) to the chainDump.py script. Move to using the new format in the CI reference comparison with the following changes:

  1. Add a new script chainComp.py analysing the differences, providing human-readable concise summary and creating a git patch file which can be easily applied locally. When running in CI, the contents of the patch are printed to stdout such that they can be easily copied and used locally.
  2. Update the two test relying on reference comparison to use the new format and the new script:
    • Define a step in TrigValSteering.CheckSteps
    • Use the new step in the test scripts
    • Update the two references
  3. Add a feature to runTrigART.py to print the chainComp.py summary and patch file path in case of reference mismatch.

Jira: ATR-21386

Example chainComp output:
chainComp INFO     Test file: ref_data_v1Dev_build.new
chainComp INFO     Reference file: /workspaces/build/x86_64-centos7-gcc8-opt/data/TriggerTest/ref_data_v1Dev_build.ref
chainComp INFO     Found 1 new chain added:
chainComp INFO       HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20
chainComp INFO     Found 1 chain removed:
chainComp INFO       HLT_2mu5_bUpsimumu_L12MU4
chainComp INFO     Found 3 chains with count differences:
chainComp INFO       HLT_e3_etcut1step_g5_etcut_L12EM3:
chainComp INFO         eventCount: 8 -> 10
chainComp INFO         stepCounts:
chainComp INFO           1: 8 -> 10
chainComp INFO           2: 8 -> 10
chainComp INFO         stepFeatures:
chainComp INFO           1: 20 -> 23
chainComp INFO           2: 20 -> 23
chainComp INFO       HLT_e5_etcut_L1EM3:
chainComp INFO         stepFeatures:
chainComp INFO           1: 123 -> 148
chainComp INFO       HLT_g5_tight_L1EM3:
chainComp INFO         eventCount: 0 -> 1
chainComp INFO         stepCounts:
chainComp INFO           3: 0 -> 1
chainComp INFO         stepFeatures:
chainComp INFO           3: 0 -> 1
chainComp ERROR    Trigger counts differ from the reference. If the above differences are intended, update the reference
chainComp INFO     Patch file created. To apply, run in the athena source directory:
chainComp INFO     git apply /workspaces/build/runTrigART/results/runTrigART/test_trig_data_v1Dev_build/ref_data_v1Dev_build.patch
chainComp INFO     Then check with git diff and, if everything is correct, add and commit
Corresponding example patch file:
diff --git a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
--- a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
+++ b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
@@ -116,2 +116,10 @@
     1: 2
+HLT_2mu5_bUpsimumu_L12MU4:
+  eventCount: 0
+  stepCounts:
+    0: 1
+    1: 1
+  stepFeatures:
+    0: 4
+    1: 2
 HLT_2mu4_muonqual_L12MU4:
@@ -323,11 +331,11 @@
 HLT_e3_etcut1step_g5_etcut_L12EM3:
-  eventCount: 10
+  eventCount: 8
   stepCounts:
     0: 10
-    1: 10
-    2: 10
+    1: 8
+    2: 8
   stepFeatures:
     0: 23
-    1: 23
-    2: 23
+    1: 20
+    2: 20
 HLT_e3_etcut_L1EM3:
@@ -353,3 +361,3 @@
     0: 41
-    1: 148
+    1: 123
     2: 49
@@ -597,3 +605,3 @@
 HLT_g5_tight_L1EM3:
-  eventCount: 1
+  eventCount: 0
   stepCounts:
@@ -602,3 +610,2 @@
     2: 9
-    3: 1
   stepFeatures:
@@ -607,3 +614,2 @@
     2: 29
-    3: 1
 HLT_j0_aggSEP1000htSEP30etSEP0eta320_L1J20:
@@ -740,12 +746,2 @@
     1: 5
-HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20:
-  eventCount: 5
-  stepCounts:
-    0: 5
-    1: 5
-    2: 5
-  stepFeatures:
-    0: 5
-    1: 5
-    2: 5
 HLT_j45_sktc_nojcalib_L1J20:
Edited by Rafal Bielski

Merge request reports