Draft: Run Allen checkers in Moore

Add functionality to run the Allen (standalone) checkers within Moore, by using Gaudi-Allen algorithms.

• Re-work allen_gaudi_... options scripts to actually call Gaudi-Allen algorithms for the reconstruction rather than the full allen sequence
• Add the Allen validators to the respective validation sequences
• Use the newly available default_VeloCluster_source function to specify whether Retina or VP clusters are used as input (was not included in !2786 (merged) for the allen_gaudi_... option scripts.)

It is working on the velo reconstruction for now and gives the following output:

velo_validator validation:
TrackChecker output                               :       145/    20963   0.69% ghosts
01_velo                                           :      8782/     8930  98.34% ( 98.41%),       181 (  2.02%) clones, pur  99.70%, hit eff  96.86%
02_long                                           :      5228/     5271  99.18% ( 99.15%),        80 (  1.51%) clones, pur  99.76%, hit eff  97.79%
03_long_P>5GeV                                    :      3625/     3640  99.59% ( 99.57%),        42 (  1.15%) clones, pur  99.80%, hit eff  98.36%
04_long_strange                                   :       280/      286  97.90% ( 97.11%),         6 (  2.10%) clones, pur  99.43%, hit eff  97.20%
05_long_strange_P>5GeV                            :       139/      140  99.29% ( 99.83%),         3 (  2.11%) clones, pur  99.34%, hit eff  96.92%
06_long_fromB                                     :         7/        7 100.00% (100.00%),         0 (  0.00%) clones, pur 100.00%, hit eff  99.05%
07_long_fromB_P>5GeV                              :         7/        7 100.00% (100.00%),         0 (  0.00%) clones, pur 100.00%, hit eff  99.05%
08_long_electrons                                 :       482/      494  97.57% ( 98.28%),        12 (  2.43%) clones, pur  98.10%, hit eff  96.27%

TransposeRawBanks_38559b47          SUCCESS Booked 1 Histogram(s) : 1D=1 
VeloTrackChecker_5430d5f0              INFO Results
VeloTrackChecker_5430d5f0              INFO **** Velo                                      20963 tracks including            145 ghosts [ 0.69 %], Event average  0.60 % ****
VeloTrackChecker_5430d5f0              INFO   01_velo                                  :    8782 from     8930 [ 98.34 %]    181 clones [ 2.02 %], purity: 99.70 %, hitEff: 96.86 %, hitEffFirst3: 96.98 %, hitEffLast: 96.66 %
VeloTrackChecker_5430d5f0              INFO   02_long                                  :    5228 from     5271 [ 99.18 %]     80 clones [ 1.51 %], purity: 99.76 %, hitEff: 97.79 %, hitEffFirst3: 97.85 %, hitEffLast: 97.68 %
VeloTrackChecker_5430d5f0              INFO   03_long_P>5GeV                           :    3625 from     3640 [ 99.59 %]     42 clones [ 1.15 %], purity: 99.80 %, hitEff: 98.36 %, hitEffFirst3: 98.39 %, hitEffLast: 98.31 %
VeloTrackChecker_5430d5f0              INFO   04_long_strange                          :     304 from      311 [ 97.75 %]      7 clones [ 2.25 %], purity: 99.23 %, hitEff: 96.74 %, hitEffFirst3: 96.89 %, hitEffLast: 96.52 %
VeloTrackChecker_5430d5f0              INFO   05_long_strange_P>5GeV                   :     149 from      151 [ 98.68 %]      3 clones [ 1.97 %], purity: 99.11 %, hitEff: 96.86 %, hitEffFirst3: 96.49 %, hitEffLast: 97.22 %
VeloTrackChecker_5430d5f0              INFO   06_long_fromB                            :       7 from        7 [100.00 %]      0 clones [ 0.00 %], purity:100.00 %, hitEff: 99.05 %, hitEffFirst3: 95.24 %, hitEffLast:100.00 %
VeloTrackChecker_5430d5f0              INFO   06_long_fromD                            :      65 from       66 [ 98.48 %]      1 clones [ 1.52 %], purity: 99.53 %, hitEff: 97.38 %, hitEffFirst3: 96.46 %, hitEffLast: 97.80 %
VeloTrackChecker_5430d5f0              INFO   07_long_fromB_P>5GeV                     :       7 from        7 [100.00 %]      0 clones [ 0.00 %], purity:100.00 %, hitEff: 99.05 %, hitEffFirst3: 95.24 %, hitEffLast:100.00 %
VeloTrackChecker_5430d5f0              INFO   07_long_fromD_P>5GeV                     :      46 from       47 [ 97.87 %]      0 clones [ 0.00 %], purity: 99.76 %, hitEff: 98.42 %, hitEffFirst3: 97.10 %, hitEffLast: 99.06 %
VeloTrackChecker_5430d5f0              INFO   08_long_electrons                        :     482 from      494 [ 97.57 %]     12 clones [ 2.43 %], purity: 98.10 %, hitEff: 96.27 %, hitEffFirst3: 94.40 %, hitEffLast: 97.18 %
VeloTrackChecker_5430d5f0              INFO   11_long_fromB_P>3GeV_Pt>0.5GeV           :       6 from        6 [100.00 %]      0 clones [ 0.00 %], purity:100.00 %, hitEff:100.00 %, hitEffFirst3:100.00 %, hitEffLast:100.00 %
VeloTrackChecker_5430d5f0              INFO   11_long_fromD_P>3GeV_Pt>0.5GeV           :      42 from       43 [ 97.67 %]      0 clones [ 0.00 %], purity: 99.74 %, hitEff: 98.43 %, hitEffFirst3: 97.62 %, hitEffLast: 98.95 %
VeloTrackChecker_5430d5f0              INFO   11_long_strange_P>3GeV_Pt>0.5GeV         :      78 from       81 [ 96.30 %]      0 clones [ 0.00 %], purity: 99.37 %, hitEff: 98.06 %, hitEffFirst3: 97.86 %, hitEffLast: 98.53 %
VeloTrackChecker_5430d5f0              INFO   12_UT_long_fromB_P>3GeV_Pt>0.5GeV        :       6 from        6 [100.00 %]      0 clones [ 0.00 %], purity:100.00 %, hitEff:100.00 %, hitEffFirst3:100.00 %, hitEffLast:100.00 %
VeloTrackChecker_5430d5f0              INFO

Outstanding items:

• Fix error encountered when running the PrTrackerDumper including UT hits inside a validation script (workaround for now by ordering algorithms such that the crash does not occur, see #624)
• Re-work the veloUT, forward, seeding-matching, downstream, PV scripts to use Gaudi-Allen algorithms and call the Allen validators
• Track down differences between Allen and Rec checkers
• Resolve any differences found
• Compare Allen and Rec checker outputs against each other within qmtest

Goes together with Allen!1258

Hlt/Hlt1Conf/tests/options/allen_fwd_tracks_tos.py

@@ -11,30 +11,42 @@
 from Moore import options
 from Moore.config import get_allen_hlt1_decision_ids, allen_detectors
 from RecoConf.hlt1_allen import (
-    make_allen_sel_reports, make_allen_forward_tracks,
-    call_allen_decision_logger, make_allen_decision, allen_gaudi_config)
+    make_allen_sel_reports, call_allen_decision_logger, make_allen_decision,
+    allen_gaudi_config, get_v1keyed_v1_v2_tracks_from_v3_tracks)
 from PyConf.application import configure_input, configure
 from PyConf.control_flow import CompositeNode, NodeLogic
 from Allen.config import setup_allen_non_event_data_service
 
 
-def make_tos_filter(line_name):
+def make_tos_filter(line_name, hlt1_config):
     from PyConf.Algorithms import TOSFilter__v1__Track as TOSFilter
     from PyConf.Tools import ParticleTisTos
+    from PyConf.Algorithms import GaudiAllenMEBasicParticlesToV3Tracks
+    from RecoConf.core_algorithms import make_unique_id_generator
+
+    allen_reco = hlt1_config['reconstruction']
+    forward_tracks_v3 = GaudiAllenMEBasicParticlesToV3Tracks(
+        allen_tracks_mec=allen_reco['long_track_particles']
+        ['dev_multi_event_basic_particles'],
+        InputUniqueIDGenerator=make_unique_id_generator()).OutputTracks
+
+    converted_tracks = get_v1keyed_v1_v2_tracks_from_v3_tracks(
+        forward_tracks_v3)
 
     return TOSFilter(
         name=line_name + 'TOS',
-        InputContainer=make_allen_forward_tracks()['v1'],
+        InputContainer=converted_tracks['v1'],
         SelReports=make_allen_sel_reports(),
         DecisionPattern=line_name,
         ParticleTisTosTool=ParticleTisTos(
             TOSFracVP=1.0, TOSFracFT=1.0, TOSFracMuon=1.0))
 
 
-def setup_tos_node():
+def setup_tos_node(hlt1_config):
     hlt1_line_ids = get_allen_hlt1_decision_ids()
     filt_algs = [
-        make_tos_filter(line_name) for line_name in hlt1_line_ids.keys()
+        make_tos_filter(line_name, hlt1_config)
+        for line_name in hlt1_line_ids.keys()
     ]
 
     return CompositeNode(
@@ -56,7 +68,7 @@ def run_test():
         bank_types=detectors)
 
     # Get the configured TOS filters
-    tos_node = setup_tos_node()
+    tos_node = setup_tos_node(hlt1_config)
 
     # Make a complete control flow
     test_node = CompositeNode(