Draft: Run Allen checkers in Moore

Add functionality to run the Allen (standalone) checkers within Moore, by using Gaudi-Allen algorithms.

• add MCRawBankToMCEvent Gaudi algorithm, which provides the MC information needed by the Allen checkers in the correct format
• re-work the Allen python configuration to enable calling the validators from Allen standalone and Gaudi-Allen
• Fix PrTrackerDumper for FT hits - the Allen standalone MC checker will work again for Scifi tracks now
• the detector list is no longer specified specifically for the dd4hep build here

More description concerning the Moore part can be found in the corresponding Moore MR.

Goes together with Moore!2462

added RTA label

changed the description

mentioned in merge request Moore!2462

added MC checking label

added 42 commits

• 33024c47...3f0e97f6 - 36 commits from branch master
• 65f1d576 - add MCRawBankToMCEvent
• 4f06938b - update velo validation python configuration to take mc_events as input
• 8ea89ab2 - provide CheckerInvoker in RuntimeOptions when calling Allen from Moore
• af7fdebb - cleanup unused legacy function to read MC information from binary files
• aba163df - updateveloUT validation to take mc_events as input
• 1d8d1de5 - add ut_banks as output to decode_ut to reproduce UT hit crash

Compare with previous version

mentioned in issue Moore#624

added 3 commits

• 9a8648a2 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• ebbf1465 - update long validation for call with Gaudi-Allen
• baed1ec2 - Added GaudiAllenMELongTracksToV3Tracks track converter

Compare with previous version

added 78 commits

• baed1ec2...2436f91e - 68 earlier commits
• be762d4c - add MCRawBankToMCEvent
• 46512eb9 - update velo validation python configuration to take mc_events as input
• aa8b535c - provide CheckerInvoker in RuntimeOptions when calling Allen from Moore
• a572db16 - cleanup unused legacy function to read MC information from binary files
• 6948cbba - updateveloUT validation to take mc_events as input
• 518d40c7 - add ut_banks as output to decode_ut to reproduce UT hit crash
• fde20fb2 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• 658ffc69 - update long validation for call with Gaudi-Allen
• 71d292e5 - Added GaudiAllenMELongTracksToV3Tracks track converter
• 8cd26015 - adjust veloUT and long MC categories to match those in the PrChecker

Compare with previous version

marked this merge request as ready

added 145 commits

• 8cd26015...08c9f50a - 135 commits from branch master
• 238821fe - add MCRawBankToMCEvent
• 43cebc6e - update velo validation python configuration to take mc_events as input
• 63c4ff68 - provide CheckerInvoker in RuntimeOptions when calling Allen from Moore
• 3f3e3eed - cleanup unused legacy function to read MC information from binary files
• ac9b2e1f - updateveloUT validation to take mc_events as input
• d714df04 - add ut_banks as output to decode_ut to reproduce UT hit crash
• c13f5521 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• fded4f9a - update long validation for call with Gaudi-Allen
• 7f0d3eb4 - Added GaudiAllenMELongTracksToV3Tracks track converter
• a99105aa - adjust veloUT and long MC categories to match those in the PrChecker

Compare with previous version

added 1 commit

• 24be65b2 - Fixed formatting

Compare with previous version

@spradlin I have cherry-picked your commit with the converters here. Do you prefer to make a separate MR or can this go in as part of this one?

requested review from @raaij

• [2023-08-03 15:24] Validation started with lhcb-master-mr#8774
• [2023-09-05 19:24] Validation started with lhcb-master-mr#9139
• [2023-09-15 18:16] Validation started with lhcb-master-mr#9228
• [2024-01-17 13:19] Validation started with lhcb-master-mr#10150
• [2024-01-23 12:14] Validation started with lhcb-master-mr#10215
• [2024-02-09 09:36] Validation started with lhcb-master-mr#10474
• [2024-02-10 11:02] Validation started with lhcb-master-mr#10501
• [2024-02-29 11:52] Validation started with lhcb-master-mr#10812
• [2024-03-12 16:28] Validation started with lhcb-master-mr#11013
• [2024-03-18 09:02] Validation started with lhcb-master-mr#11116
• [2024-04-15 16:47] Validation started with lhcb-2024-patches-mr#93

resolved all threads

mentioned in issue Moore#627 (closed)

mentioned in issue Moore#634 (closed)

@dovombru This needs some additional work together with Moore!2462, right? Please ping me when ready to be tested.

mentioned in merge request Moore!2439 (closed)

mentioned in issue Moore#652 (closed)

added 45 commits

• 24be65b2...1d62c43a - 33 commits from branch master
• 1d62c43a...18d9d936 - 2 earlier commits
• c0c4a8e9 - provide CheckerInvoker in RuntimeOptions when calling Allen from Moore
• e1d06016 - cleanup unused legacy function to read MC information from binary files
• 27e854ac - updateveloUT validation to take mc_events as input
• 41edf7c5 - add ut_banks as output to decode_ut to reproduce UT hit crash
• a6a484c7 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• 379456d7 - update long validation for call with Gaudi-Allen
• 5c5dd2fe - Added GaudiAllenMELongTracksToV3Tracks track converter
• e24ef5ea - adjust veloUT and long MC categories to match those in the PrChecker
• d7ea4f32 - Fixed formatting
• abb2fdbb - fix channelID to lhcbID for SciFi LHCbID dumping of MCParticles

Compare with previous version

@lohenry it looks like !1240 (merged) changed the lhcbID dumping of the FT hits from lhcbID to channelID, i.e. the upper bits identifying the detector were no longer present. This resulted in the Allen standalone MC checkers not working anymore for the SciFi part. With this set of MRs I'm introducing a test that should catch this. But I was wondering if there was any reasoning behind it?

changed the description

added 1 commit

• 6edd27e3 - Fixed formatting

Compare with previous version

resolved all threads

added 41 commits

• 6edd27e3...5b90acf1 - 28 commits from branch master
• 5b90acf1...ffa4385a - 3 earlier commits
• 591b68cd - cleanup unused legacy function to read MC information from binary files
• b40af35d - updateveloUT validation to take mc_events as input
• 4a29320e - add ut_banks as output to decode_ut to reproduce UT hit crash
• bfd70019 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• 9d5d2049 - update long validation for call with Gaudi-Allen
• dd356eff - Added GaudiAllenMELongTracksToV3Tracks track converter
• 2babd124 - adjust veloUT and long MC categories to match those in the PrChecker
• 76396851 - Fixed formatting
• 6c91e732 - fix channelID to lhcbID for SciFi LHCbID dumping of MCParticles
• 955671cc - Fixed formatting

Compare with previous version

mentioned in issue Moore#659 (closed)

mentioned in issue Moore#669 (closed)

added needs ref update label

mentioned in issue Moore#671 (closed)

added RTA WP2 RTA WP5 impactlow prioritylow labels

assigned to @rmatev and @cagapopo

assigned to @dovombru and unassigned @rmatev and @cagapopo

Hello @dovombru, is this ready to be tested together with Moore!2462 push forward? I see it was tested at Moore!2462 (comment 7322961) but no conclusion from that test.

I see some differences in Allen ci, namely physics efficiency, but that looks like some new checks are added so a reference update is needed. Could you confirm?

added 136 commits

• 955671cc...994dc6f5 - 125 commits from branch master
• 01e92f01 - 1 earlier commit
• ac205b09 - update velo validation python configuration to take mc_events as input
• f3cec584 - provide CheckerInvoker in RuntimeOptions when calling Allen from Moore
• b4851e00 - cleanup unused legacy function to read MC information from binary files
• 41841e2a - updateveloUT validation to take mc_events as input
• 9e84c535 - add ut_banks as output to decode_ut to reproduce UT hit crash
• 3d90c4e5 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• e91dc950 - update long validation for call with Gaudi-Allen
• 507b3587 - Added GaudiAllenMELongTracksToV3Tracks track converter
• 8b0013e0 - adjust veloUT and long MC categories to match those in the PrChecker
• 5a688b49 - Fixed formatting

Compare with previous version

added 1 commit

• 2266be19 - Fix python configuration for updated functions

Compare with previous version

added 1 commit

• 98a2ed39 - Fixed formatting

Compare with previous version

@raaij this is ready for review.

resolved all threads

/ci-test Moore!2462

added ci-test-triggered label

mentioned in issue Moore#688 (closed)

added 1 commit

• c37376ff - Detector list no longer configured specifically for dd4hep

Compare with previous version

added 35 commits

• c37376ff...dc224ace - 19 commits from branch master
• dc224ace...fde2e04b - 6 earlier commits
• 78c1b2d9 - adjust Velo MC checking categories and 'fromStrangeDecay' definition to include K+
• 23c5b8c6 - update long validation for call with Gaudi-Allen
• e6a218e3 - Added GaudiAllenMELongTracksToV3Tracks track converter
• bd440c76 - adjust veloUT and long MC categories to match those in the PrChecker
• 30c9ecf1 - Fixed formatting
• e1746b08 - Fix python configuration for updated functions
• 0ef08e37 - Fixed formatting
• 487049dc - Detector list no longer configured specifically for dd4hep
• e67de25a - fix compilation after rebase
• b4230a3b - disable validator root histogram filling when compiling with Gaudi

Compare with previous version

added 1 commit

• d7731398 - Fixed formatting

Compare with previous version

changed the description

mentioned in issue #128

added 74 commits

• d7731398...30b6e450 - 57 commits from branch master
• 30b6e450...8b095ef1 - 7 earlier commits
• eb18c841 - update long validation for call with Gaudi-Allen
• 234ab5d2 - Added GaudiAllenMELongTracksToV3Tracks track converter
• ac572e5e - adjust veloUT and long MC categories to match those in the PrChecker
• a4510190 - Fixed formatting
• a650acac - Fix python configuration for updated functions
• d6c3ecbc - Fixed formatting
• f423b3f8 - Detector list no longer configured specifically for dd4hep
• 68006ff5 - fix compilation after rebase
• a19aae3c - disable validator root histogram filling when compiling with Gaudi
• 995ff467 - Fixed formatting

Compare with previous version

added 1 commit

• e104addb - fixes after rebase

Compare with previous version

unassigned @dovombru