New trigger counts reference comparison tools and format

New format for trigger counts summary was added in !37918 (merged) to the chainDump.py script. Move to using the new format in the CI reference comparison with the following changes:

1. Add a new script chainComp.py analysing the differences, providing human-readable concise summary and creating a git patch file which can be easily applied locally. When running in CI, the contents of the patch are printed to stdout such that they can be easily copied and used locally.
2. Update the two test relying on reference comparison to use the new format and the new script:
   • Define a step in TrigValSteering.CheckSteps
   • Use the new step in the test scripts
   • Update the two references
3. Add a feature to runTrigART.py to print the chainComp.py summary and patch file path in case of reference mismatch.

Jira: ATR-21386

Example chainComp output:

chainComp INFO     Test file: ref_data_v1Dev_build.new
chainComp INFO     Reference file: /workspaces/build/x86_64-centos7-gcc8-opt/data/TriggerTest/ref_data_v1Dev_build.ref
chainComp INFO     Found 1 new chain added:
chainComp INFO       HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20
chainComp INFO     Found 1 chain removed:
chainComp INFO       HLT_2mu5_bUpsimumu_L12MU4
chainComp INFO     Found 3 chains with count differences:
chainComp INFO       HLT_e3_etcut1step_g5_etcut_L12EM3:
chainComp INFO         eventCount: 8 -> 10
chainComp INFO         stepCounts:
chainComp INFO           1: 8 -> 10
chainComp INFO           2: 8 -> 10
chainComp INFO         stepFeatures:
chainComp INFO           1: 20 -> 23
chainComp INFO           2: 20 -> 23
chainComp INFO       HLT_e5_etcut_L1EM3:
chainComp INFO         stepFeatures:
chainComp INFO           1: 123 -> 148
chainComp INFO       HLT_g5_tight_L1EM3:
chainComp INFO         eventCount: 0 -> 1
chainComp INFO         stepCounts:
chainComp INFO           3: 0 -> 1
chainComp INFO         stepFeatures:
chainComp INFO           3: 0 -> 1
chainComp ERROR    Trigger counts differ from the reference. If the above differences are intended, update the reference
chainComp INFO     Patch file created. To apply, run in the athena source directory:
chainComp INFO     git apply /workspaces/build/runTrigART/results/runTrigART/test_trig_data_v1Dev_build/ref_data_v1Dev_build.patch
chainComp INFO     Then check with git diff and, if everything is correct, add and commit

Corresponding example patch file:

diff --git a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
--- a/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
+++ b/Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
@@ -116,2 +116,10 @@
     1: 2
+HLT_2mu5_bUpsimumu_L12MU4:
+  eventCount: 0
+  stepCounts:
+    0: 1
+    1: 1
+  stepFeatures:
+    0: 4
+    1: 2
 HLT_2mu4_muonqual_L12MU4:
@@ -323,11 +331,11 @@
 HLT_e3_etcut1step_g5_etcut_L12EM3:
-  eventCount: 10
+  eventCount: 8
   stepCounts:
     0: 10
-    1: 10
-    2: 10
+    1: 8
+    2: 8
   stepFeatures:
     0: 23
-    1: 23
-    2: 23
+    1: 20
+    2: 20
 HLT_e3_etcut_L1EM3:
@@ -353,3 +361,3 @@
     0: 41
-    1: 148
+    1: 123
     2: 49
@@ -597,3 +605,3 @@
 HLT_g5_tight_L1EM3:
-  eventCount: 1
+  eventCount: 0
   stepCounts:
@@ -602,3 +610,2 @@
     2: 9
-    3: 1
   stepFeatures:
@@ -607,3 +614,2 @@
     2: 29
-    3: 1
 HLT_j0_aggSEP1000htSEP30etSEP0eta320_L1J20:
@@ -740,12 +746,2 @@
     1: 5
-HLT_j45_pf_subjesgscIS_ftf_boffperf_split_L1J20:
-  eventCount: 5
-  stepCounts:
-    0: 5
-    1: 5
-    2: 5
-  stepFeatures:
-    0: 5
-    1: 5
-    2: 5
 HLT_j45_sktc_nojcalib_L1J20:

Jenkins please retry a build

This merge request affects 7 files:

• Trigger/TrigValidation/TrigAnalysisTest/share/ref_RDOtoRDOTrig_v1Dev_build.ref
• Trigger/TrigValidation/TrigAnalysisTest/test/test_trigAna_RDOtoRDOTrig_v1Dev_build.py
• Trigger/TrigValidation/TrigValTools/bin/chainComp.py
• Trigger/TrigValidation/TrigValTools/bin/runTrigART.py
• Trigger/TrigValidation/TrigValTools/python/TrigValSteering/CheckSteps.py
• Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
• Trigger/TrigValidation/TriggerTest/test/test_trig_data_v1Dev_build.py

Adding @okumura ,@rbielski ,@jpanduro as watchers

added Trigger master review-pending-level-1 labels

removed review-pending-level-1 label

CI Result SUCCESS (hash 1a9cd22c)

	Athena	AthSimulation	AthGeneration	AnalysisBase
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23257]

Hi @rbielski, this looks awesome!!

Looking at the pending merge requests, this seems like a good time to make this change. There doesn't seem to be anything else ready to go or anything important pending.

unmarked as a Work In Progress

Thank you @hrussell and @markowen for the comments (here / in Jira)! I resolved conflicts and pushed an update, but the new commits didn't appear in the MR yet. Anyway, I marked as ready for review and hopefully the CI runs soon.

added 77 commits

• 1a9cd22c...eb3b5bb0 - 76 commits from branch atlas:master
• 3661d353 - New trigger counts reference comparison tools and format

Compare with previous version

This merge request affects 7 files:

• Trigger/TrigValidation/TrigAnalysisTest/share/ref_RDOtoRDOTrig_v1Dev_build.ref
• Trigger/TrigValidation/TrigAnalysisTest/test/test_trigAna_RDOtoRDOTrig_v1Dev_build.py
• Trigger/TrigValidation/TrigValTools/bin/chainComp.py
• Trigger/TrigValidation/TrigValTools/bin/runTrigART.py
• Trigger/TrigValidation/TrigValTools/python/TrigValSteering/CheckSteps.py
• Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
• Trigger/TrigValidation/TriggerTest/test/test_trig_data_v1Dev_build.py

Adding @okumura ,@rbielski ,@jpanduro as watchers

added review-pending-level-1 label

CI Result SUCCESS (hash 3661d353)

	Athena	AthSimulation	AthGeneration	AnalysisBase
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23318]

Restarting CI to test https://gitlab.cern.ch/atlas-sit/CI/-/merge_requests/178

Jenkins please retry a build

This merge request affects 7 files:

• Trigger/TrigValidation/TrigAnalysisTest/share/ref_RDOtoRDOTrig_v1Dev_build.ref
• Trigger/TrigValidation/TrigAnalysisTest/test/test_trigAna_RDOtoRDOTrig_v1Dev_build.py
• Trigger/TrigValidation/TrigValTools/bin/chainComp.py
• Trigger/TrigValidation/TrigValTools/bin/runTrigART.py
• Trigger/TrigValidation/TrigValTools/python/TrigValSteering/CheckSteps.py
• Trigger/TrigValidation/TriggerTest/share/ref_data_v1Dev_build.ref
• Trigger/TrigValidation/TriggerTest/test/test_trig_data_v1Dev_build.py

Adding @okumura ,@rbielski ,@jpanduro as watchers

added changes-trigger-counts label

It works Now atlasbot adds changes-trigger-counts whenever the references are modified

CI Result SUCCESS (hash 3661d353)

	Athena	AthSimulation	AthGeneration	AnalysisBase
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23326]

added review-approved label and removed review-pending-level-1 label