New trigger counts reference comparison tools and format
New format for trigger counts summary was added in !37918 (merged) to the chainDump.py
script. Move to using the new format in the CI reference comparison with the following changes:
- Add a new script
chainComp.py
analysing the differences, providing human-readable concise summary and creating a git patch file which can be easily applied locally. When running in CI, the contents of the patch are printed to stdout such that they can be easily copied and used locally. - Update the two test relying on reference comparison to use the new format and the new script:
- Define a step in TrigValSteering.CheckSteps
- Use the new step in the test scripts
- Update the two references
- Add a feature to
runTrigART.py
to print thechainComp.py
summary and patch file path in case of reference mismatch.
Jira: ATR-21386
Example chainComp output:
chainComp INFO Test file: ref_data_v1Dev_build.new
chainComp INFO Reference file: /workspaces/build/x86_64-centos7-gcc8-opt/data/TriggerTest/ref_data_v1Dev_build.ref
chainComp INFO Found 1 new chain added:
chainComp INFO HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20
chainComp INFO Found 1 chain removed:
chainComp INFO HLT_2mu5_bUpsimumu_L12MU4
chainComp INFO Found 3 chains with count differences:
chainComp INFO HLT_e3_etcut1step_g5_etcut_L12EM3:
chainComp INFO eventCount: 8 -> 10
chainComp INFO stepCounts:
chainComp INFO 1: 8 -> 10
chainComp INFO 2: 8 -> 10
chainComp INFO stepFeatures:
chainComp INFO 1: 20 -> 23
chainComp INFO 2: 20 -> 23
chainComp INFO HLT_e5_etcut_L1EM3:
chainComp INFO stepFeatures:
chainComp INFO 1: 123 -> 148
chainComp INFO HLT_g5_tight_L1EM3:
chainComp INFO eventCount: 0 -> 1
chainComp INFO stepCounts:
chainComp INFO 3: 0 -> 1
chainComp INFO stepFeatures:
chainComp INFO 3: 0 -> 1
chainComp ERROR Trigger counts differ from the reference. If the above differences are intended, update the reference
chainComp INFO Patch file created. To apply, run in the athena source directory:
chainComp INFO git apply /workspaces/build/runTrigART/results/runTrigART/test_trig_data_v1Dev_build/ref_data_v1Dev_build.patch
chainComp INFO Then check with git diff and, if everything is correct, add and commit
Corresponding example patch file:
diff --git a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
--- a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
+++ b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
@@ -116,2 +116,10 @@
1: 2
+HLT_2mu5_bUpsimumu_L12MU4:
+ eventCount: 0
+ stepCounts:
+ 0: 1
+ 1: 1
+ stepFeatures:
+ 0: 4
+ 1: 2
HLT_2mu4_muonqual_L12MU4:
@@ -323,11 +331,11 @@
HLT_e3_etcut1step_g5_etcut_L12EM3:
- eventCount: 10
+ eventCount: 8
stepCounts:
0: 10
- 1: 10
- 2: 10
+ 1: 8
+ 2: 8
stepFeatures:
0: 23
- 1: 23
- 2: 23
+ 1: 20
+ 2: 20
HLT_e3_etcut_L1EM3:
@@ -353,3 +361,3 @@
0: 41
- 1: 148
+ 1: 123
2: 49
@@ -597,3 +605,3 @@
HLT_g5_tight_L1EM3:
- eventCount: 1
+ eventCount: 0
stepCounts:
@@ -602,3 +610,2 @@
2: 9
- 3: 1
stepFeatures:
@@ -607,3 +614,2 @@
2: 29
- 3: 1
HLT_j0_aggSEP1000htSEP30etSEP0eta320_L1J20:
@@ -740,12 +746,2 @@
1: 5
-HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20:
- eventCount: 5
- stepCounts:
- 0: 5
- 1: 5
- 2: 5
- stepFeatures:
- 0: 5
- 1: 5
- 2: 5
HLT_j45_sktc_nojcalib_L1J20:
Edited by Rafal Bielski