Draft: Match Upstream Muon, DiMuonTrackEfficiencyLine and small EventModel changes on top of MES
The changes introduced here can be factored in two big categories:
- Optimisation/speed-up of MatchUpstreamMuon algorithm
- Slight changes on event model and sequence to account VeloUT/Forward tracks vertices.
A custom sequence called hlt1_pp_mum (mum = MatchUpstreamMuon) contains the following algorithms:
-
FakeInput: Builds self referential ut_indices needed to buildUTTrackswithSciFiconstructor -
MuonFilter: Selects events containing tracks statisfying a certain criterio by building a new event list -
MuonTrackType: Filters out low p tracks to minimize branching in MatchUpstreamMuon -
MatchUpstreamMuon: SelectsVeloUTtracks that match certain Muon hits -
KalmanVeloOnlyMF: Reconfiguration ofKalmanVeloOnlyalgorithm applied toVeloUTin events passing the MuonFilter FilterTracksMFFitSecondaryVerticesMF-
JoinVertices: The idea here is to keep allSVsin the same array. But this would mean deeper changes in event model...
Also old hardcoded version of these implementations is removed since it is no longer needed.
To being able to use these algorithms some modifications were needed. This implied constructing UTTracks as SciFi Tracks.
This is how it looks like from the configuration point of view:
filter_tracks_mf = filter_tracks_t(
name='filter_tracks_mf',
host_number_of_events_t=initialize_lists.host_number_of_events_t(),
dev_kf_tracks_1_t=kalman_velo_only.dev_kf_tracks_t(),
dev_offsets_forward_tracks_1_t=prefix_sum_forward_tracks.
dev_output_buffer_t(),
dev_offsets_scifi_track_hit_number_1_t=prefix_sum_scifi_track_hit_number
.dev_output_buffer_t(),
dev_scifi_qop_1_t=scifi_consolidate_tracks.dev_scifi_qop_t(),
dev_scifi_states_1_t=scifi_consolidate_tracks.dev_scifi_states_t(),
dev_scifi_track_ut_indices_1_t=scifi_consolidate_tracks.
dev_scifi_track_ut_indices_t(),
dev_kalman_pv_ipchi2_1_t=kalman_velo_only.dev_kalman_pv_ipchi2_t(),
dev_kf_tracks_2_t=kalman_velo_only_mf.dev_kf_tracks_t(),
dev_offsets_forward_tracks_2_t=prefix_sum_ut_tracks.
dev_output_buffer_t(),
dev_offsets_scifi_track_hit_number_2_t=prefix_sum_ut_track_hit_number.
dev_output_buffer_t(),
dev_scifi_qop_2_t=ut_consolidate_tracks.dev_ut_qop_t(),
dev_scifi_states_2_t=scifi_consolidate_tracks.dev_scifi_states_t(),
dev_scifi_track_ut_indices_2_t=fake_input.fake_ut_indices_t(),
dev_kalman_pv_ipchi2_2_t=kalman_velo_only_mf.dev_kalman_pv_ipchi2_t(),
dev_multi_fit_vertices_t=pv_beamline_multi_fitter.
dev_multi_fit_vertices_t(),
dev_number_of_multi_fit_vertices_t=pv_beamline_multi_fitter.
dev_number_of_multi_fit_vertices_t(),
dev_number_of_events_t=initialize_lists.dev_number_of_events_t(),
dev_event_list_t=event_list_mf)// Consolidated SciFi tracks.
SciFi::Consolidated::ConstTracks scifi_tracks_1 {parameters.dev_atomics_scifi_1,
parameters.dev_scifi_track_hit_number_1,
parameters.dev_scifi_qop_1,
parameters.dev_scifi_states_1,
parameters.dev_scifi_track_ut_indices_1,
event_number,
number_of_events};
const unsigned event_tracks_offset_1 = scifi_tracks_1.tracks_offset(event_number);
const unsigned n_scifi_tracks_1 = scifi_tracks_1.number_of_tracks(event_number);
// Consolidated SciFi tracks.
SciFi::Consolidated::ConstTracks scifi_tracks_2 {
parameters.dev_atomics_scifi_2,
parameters.dev_scifi_track_hit_number_2,
parameters.dev_scifi_qop_2,
parameters.dev_scifi_states_2,
parameters.dev_scifi_track_ut_indices_2,
event_number,
number_of_events}; // Maybe the more fishy part is that 'fake' SciFi tracks are built with proper SciFi states - which doesn't pose any problem but it's not ideal. Self referencing UT indices are built by FakeInput kernel.
To validate that regular sequence still works with this reconfiguration this is the throughput test in this branch as of 27/11/2020:
GeForce RTX 3090 │██████████████████████████████████████████████ 230.23 kHz
GeForce RTX 3080 │██████████████████████████████████████ 194.79 kHz
Quadro RTX 6000 │█████████████████████████████████ 166.73 kHz
GeForce RTX 2080 Ti │███████████████████████████████ 158.09 kHz
Tesla V100-PCIE-32GB │█████████████████████████████ 147.06 kHz
Intel Xeon E5-2630 v4 │▌ 4.46 kHz
┼────┴────┼────┴────┼────┴────┼────┴────┼────┴────┼
0 50 100 150 200 250 Throughput running w/o UT cut and MatchUpstreamMuon sequence:
GeForce RTX 3090 │██████████████████████████████████████████████ 192.37 kHz
GeForce RTX 3080 │███████████████████████████████████████ 162.62 kHz
Quadro RTX 6000 │███████████████████████████████████ 146.23 kHz
GeForce RTX 2080 Ti │█████████████████████████████████ 139.23 kHz
Tesla V100-PCIE-32GB │█████████████████████████████ 122.17 kHz
Intel Xeon E5-2630 v4 │▌ 3.77 kHz
┼─────┴─────┼─────┴─────┼─────┴─────┼─────┴─────┼
0 50 100 150 200 Throughput running with UT cut and MatchUpstreamMuon sequence:
GeForce RTX 3090 │████████████████████████████████████████████ 224.98 kHz
GeForce RTX 3080 │█████████████████████████████████████ 185.23 kHz
Quadro RTX 6000 │████████████████████████████████ 163.76 kHz
GeForce RTX 2080 Ti │███████████████████████████████ 155.90 kHz
Tesla V100-PCIE-32GB │███████████████████████████ 137.78 kHz
Intel Xeon E5-2630 v4 │▌ 4.47 kHz
┼────┴────┼────┴────┼────┴────┼────┴────┼────┴────┼
0 50 100 150 200 250 Of course these numbers may change if cuts on MuonFilter are applied. In the end, this goes down to what use one may give to the algorithm.
This is how checkers look for hlt1_pp_default (MatchUpstreamMuon not running so VeloUT track checker is giving random results) :
Requested options:
folder containing data directories (-f, --folder): /home/acasaisv/hltcuda/mag_down/
folder containing detector configuration (-g, --geometry): ../input/detector_configuration/down/
comma-separated list of MDF files to use as input (--mdf):
comma-separated list of MEP files to use as input (--mep):
Transpose MEPs instead of decoding from MEP layout directly (--transpose-mep): 0
path to json file containing values of configurable algorithm constants (--configuration): Sequence.json
show status of buffer and socket (--print-status): 0
show current algorithm configuration (--print-config): 0
write current algorithm configuration to file (--write-configuration): 0
number of events to process (-n, --number-of-events): 1000
number of input slices to allocate (-s, --number-of-slices): 0
number of events per slice (--events-per-slice): 1000
number of threads / streams (-t, --threads): 1
number of repetitions per thread / stream (-r, --repetitions): 1
run validation / checkers (-c, --validate): 1
memory to reserve on the device per thread / stream (megabytes) (-m, --memory): 1000
memory to reserve on the host per thread / stream (megabytes) (--host-memory): 200
verbosity [0-5] (-v, --verbosity): 3
print memory usage (-p, --print-memory): 0
import forward tracks dumped from Brunel (-i, --import-tracks):
offload part of the computation to CPU (--cpu-offload): 1
Write selected event to output file (--output-file):
select device to use (--device): 0, GeForce RTX 2080 Ti
Runs the program indefinitely (--non-stop): 0
Read events with MPI (--with-mpi):
Size of MPI sliding window (--mpi-window-size): 4
Number of MPI network slices (--mpi-number-of-slices): 6
Whether to insert random memory failures (0: off 1-15: rate of 1 in 2^N) (--inject-mem-fail): 0
Number of seconds between writes of the monitoring histograms (0: off) (--monitoring-save-period): 0
Ignore signals to update non-event data with each run change (--disable-run-changes): 1
Setting number of slices to 2
Starting timer for throughput measurement
Input complete
Checked 1000 events
Processing complete
Velo tracks:
TrackChecker output : 1921/ 226866 0.85% ghosts
01_velo : 93943/ 95311 98.56% ( 98.63%), 2022 ( 2.11%) clones, pur 99.72%, hit eff 96.64%
02_long : 53585/ 53895 99.42% ( 99.45%), 799 ( 1.47%) clones, pur 99.84%, hit eff 97.87%
03_long_P>5GeV : 33212/ 33309 99.71% ( 99.71%), 380 ( 1.13%) clones, pur 99.85%, hit eff 98.44%
04_long_strange : 2396/ 2440 98.20% ( 98.25%), 43 ( 1.76%) clones, pur 99.46%, hit eff 96.95%
05_long_strange_P>5GeV : 1102/ 1118 98.57% ( 98.23%), 13 ( 1.17%) clones, pur 99.37%, hit eff 97.79%
06_long_fromB : 248/ 248 100.00% (100.00%), 0 ( 0.00%) clones, pur 99.65%, hit eff 99.19%
07_long_fromB_P>5GeV : 183/ 183 100.00% (100.00%), 0 ( 0.00%) clones, pur 99.61%, hit eff 99.28%
08_long_electrons : 4135/ 4235 97.64% ( 97.54%), 118 ( 2.77%) clones, pur 98.11%, hit eff 97.08%
09_long_fromB_electrons : 18/ 18 100.00% (100.00%), 0 ( 0.00%) clones, pur 100.00%, hit eff 97.78%
10_long_fromB_electrons_P>5GeV : 11/ 11 100.00% (100.00%), 0 ( 0.00%) clones, pur 100.00%, hit eff 96.36%
Primary vertices:
REC and MC vertices matched by dz distance
MC PV is reconstructible if at least 4 tracks are reconstructed
MC PV is isolated if dz to closest reconstructible MC PV > 10.00 mm
REC and MC vertices matched by dz distance
All : 0.923 ( 4750/ 5147)
Isolated : 0.962 ( 2461/ 2558)
Close : 0.884 ( 2289/ 2589)
False rate : 0.005 ( 23/ 4773)
Real false rate : 0.005 ( 23/ 4773)
Clones : 0.000 ( 0/ 4750)
Velo+UT tracks:
TrackChecker output : 2888/ 44639 6.47% ghosts
01_velo : 39835/ 95311 41.79% ( 41.81%), 282 ( 0.70%) clones, pur 99.49%, hit eff 96.59%
02_velo+UT : 39752/ 82897 47.95% ( 47.98%), 282 ( 0.70%) clones, pur 99.51%, hit eff 96.59%
03_velo+UT_P>5GeV : 28075/ 40031 70.13% ( 70.22%), 196 ( 0.69%) clones, pur 99.62%, hit eff 97.45%
04_velo+notLong : 8047/ 41416 19.43% ( 19.33%), 48 ( 0.59%) clones, pur 99.26%, hit eff 95.37%
05_velo+UT+notLong : 7973/ 29661 26.88% ( 26.79%), 48 ( 0.60%) clones, pur 99.35%, hit eff 95.34%
06_velo+UT+notLong_P>5GeV : 4329/ 7279 59.47% ( 59.59%), 21 ( 0.48%) clones, pur 99.59%, hit eff 97.44%
07_long : 31788/ 53895 58.98% ( 58.95%), 234 ( 0.73%) clones, pur 99.56%, hit eff 96.90%
08_long_P>5GeV : 23755/ 33309 71.32% ( 71.29%), 175 ( 0.73%) clones, pur 99.62%, hit eff 97.46%
09_long_fromB : 189/ 248 76.21% ( 76.36%), 0 ( 0.00%) clones, pur 99.62%, hit eff 97.13%
10_long_fromB_P>5GeV : 157/ 183 85.79% ( 87.47%), 0 ( 0.00%) clones, pur 99.69%, hit eff 97.56%
11_long_electrons : 744/ 4235 17.57% ( 17.39%), 20 ( 2.62%) clones, pur 98.14%, hit eff 95.68%
12_long_fromB_electrons : 9/ 18 50.00% ( 50.00%), 0 ( 0.00%) clones, pur 100.00%, hit eff 95.56%
13_long_fromB_electrons_P>5GeV : 7/ 11 63.64% ( 63.64%), 0 ( 0.00%) clones, pur 100.00%, hit eff 94.29%
Muon matching:
Muon fraction in all MCPs: 11804/ 815358 0.01%
Muon fraction in MCPs to which a track(s) was matched: 803/ 41424 0.02%
Correctly identified muons with isMuon: 802/ 803 99.88%
Correctly identified muons from strange decays with isMuon: 1/ 1 100.00%
Correctly identified muons from B decays with isMuon: 71/ 71 100.00%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 40575/ 40621 99.89%
Ghost tracks identified as muon with isMuon: 2884/ 2888 99.86%
Forward tracks:
TrackChecker output : 1598/ 29323 5.45% ghosts
for P>3GeV,Pt>0.5GeV : 724/ 18032 4.02% ghosts
01_long : 26334/ 53895 48.86% ( 49.67%), 187 ( 0.71%) clones, pur 96.63%, hit eff 92.80%
02_long_P>5GeV : 21899/ 33309 65.74% ( 66.47%), 155 ( 0.70%) clones, pur 96.95%, hit eff 93.54%
03_long_strange : 800/ 2440 32.79% ( 32.51%), 4 ( 0.50%) clones, pur 95.97%, hit eff 92.35%
04_long_strange_P>5GeV : 602/ 1118 53.85% ( 53.42%), 4 ( 0.66%) clones, pur 96.46%, hit eff 93.48%
05_long_fromB : 169/ 248 68.15% ( 65.35%), 0 ( 0.00%) clones, pur 97.20%, hit eff 93.92%
06_long_fromB_P>5GeV : 152/ 183 83.06% ( 83.97%), 0 ( 0.00%) clones, pur 97.53%, hit eff 94.86%
07_long_electrons : 539/ 4235 12.73% ( 13.32%), 11 ( 2.00%) clones, pur 95.55%, hit eff 92.26%
08_long_electrons_P>5GeV : 470/ 2079 22.61% ( 23.22%), 11 ( 2.29%) clones, pur 95.70%, hit eff 92.68%
09_long_fromB_electrons : 7/ 18 38.89% ( 40.62%), 0 ( 0.00%) clones, pur 94.44%, hit eff 91.08%
10_long_fromB_electrons_P>5GeV : 6/ 11 54.55% ( 54.55%), 0 ( 0.00%) clones, pur 95.36%, hit eff 92.93%
Muon matching:
Muon fraction in all MCPs: 11804/ 815358 0.01%
Muon fraction in MCPs to which a track(s) was matched: 898/ 30943 0.03%
Correctly identified muons with isMuon: 797/ 898 88.75%
Correctly identified muons from strange decays with isMuon: 0/ 0 -nan%
Correctly identified muons from B decays with isMuon: 54/ 57 94.74%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 3364/ 30045 11.20%
Ghost tracks identified as muon with isMuon: 174/ 1598 10.89%
HLT1 rates:
Hlt1TrackMVA: 25/ 1000, ( 750.00 +/- 148.11) kHz
Hlt1TwoTrackMVA: 31/ 1000, ( 930.00 +/- 164.42) kHz
Hlt1NoBeam: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1BeamOne: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1BeamTwo: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1BothBeams: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1VeloMicroBias: 1/ 1000, ( 30.00 +/- 29.98) kHz
Hlt1ODINLumi: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1ODINNoBias: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1SingleHighPtMuon: 13/ 1000, ( 390.00 +/- 107.46) kHz
Hlt1LowPtMuon: 118/ 1000, ( 3540.00 +/- 306.05) kHz
Hlt1D2KK: 6/ 1000, ( 180.00 +/- 73.26) kHz
Hlt1D2KPi: 8/ 1000, ( 240.00 +/- 84.51) kHz
Hlt1D2PiPi: 3/ 1000, ( 90.00 +/- 51.88) kHz
Hlt1DiMuonHighMass: 188/ 1000, ( 5640.00 +/- 370.66) kHz
Hlt1DiMuonLowMass: 28/ 1000, ( 840.00 +/- 156.51) kHz
Hlt1DiMuonSoft: 0/ 1000, ( 0.00 +/- 0.00) kHz
Hlt1LowPtDiMuon: 44/ 1000, ( 1320.00 +/- 194.57) kHz
Hlt1TrackMuonMVA: 10/ 1000, ( 300.00 +/- 94.39) kHz
Hlt1GECPassthrough: 878/ 1000, (26340.00 +/- 310.49) kHz
Hlt1Passthrough: 1000/ 1000, (30000.00 +/- 0.00) kHz
Inclusive: 1000/ 1000, (30000.00 +/- 0.00) kHz
212.533616 events/s
Ran test for 4.705138 secondsFinally, the idea with Hlt1DiMuonTrackTypeLine is for this to be a control line which could be used to give a measurement of the reconstruction efficiency of SciFi tracks using a tag and probe method. This would take as input UT/SciFi vertices to which the MatchUpstreamMuon decision was fed to, as well as the isMuon, as usual.
note: Work is not finished but I would like this to be (at least partially) merged so that it keeps up to date with master changes as much as possible.
this MR substitutes !462 (closed) and probably the description has to be updated