Skip to content

Draft: Clustering for UT TELL40 decoding

Marian Stahl requested to merge mstahl_ut_cluster into master

Towards Rec#198. Throughput test added in !561 (merged).

Related: Moore!928 (closed).

This implements UT clustering in software. It will be the standard for v5 decoding without switch to single strip hits.
To avoid copying data, the UT raw banks are overwritten in place.
The cluster residuals in x look like this[1]:
x_residual_per_cluster_size.pdf

Performance Tests

Performances[1] are better across the board:
current master

UpstreamTrackChecker                   INFO Results
UpstreamTrackChecker                   INFO **** Upstream                                  33938 tracks including           1952 ghosts [ 5.75 %], Event average  5.27 % ****
UpstreamTrackChecker                   INFO   01_velo                                  :   30006 from    65861 [ 45.56 %]    474 clones [ 1.56 %], purity: 99.52 %, hitEff: 87.34 %
UpstreamTrackChecker                   INFO   02_velo+UT                               :   29945 from    57215 [ 52.34 %]    474 clones [ 1.56 %], purity: 99.54 %, hitEff: 87.33 %
UpstreamTrackChecker                   INFO   03_velo+UT_P>5GeV                        :   21376 from    29341 [ 72.85 %]    350 clones [ 1.61 %], purity: 99.63 %, hitEff: 89.06 %
UpstreamTrackChecker                   INFO   04_velo+notLong                          :    6121 from    28074 [ 21.80 %]     84 clones [ 1.35 %], purity: 99.27 %, hitEff: 85.15 %
UpstreamTrackChecker                   INFO   05_velo+UT+notLong                       :    6070 from    19887 [ 30.52 %]     84 clones [ 1.36 %], purity: 99.35 %, hitEff: 85.11 %
UpstreamTrackChecker                   INFO   06_velo+UT+notLong_P>5GeV                :    3297 from     5322 [ 61.95 %]     58 clones [ 1.73 %], purity: 99.54 %, hitEff: 89.73 %
UpstreamTrackChecker                   INFO   07_long                                  :   23885 from    37787 [ 63.21 %]    390 clones [ 1.61 %], purity: 99.59 %, hitEff: 87.89 %
UpstreamTrackChecker                   INFO   07_long_strange                          :    1292 from     2576 [ 50.16 %]     22 clones [ 1.67 %], purity: 99.26 %, hitEff: 88.01 %
UpstreamTrackChecker                   INFO   08_long_P>5GeV                           :   18089 from    24385 [ 74.18 %]    292 clones [ 1.59 %], purity: 99.64 %, hitEff: 88.95 %
UpstreamTrackChecker                   INFO   08_long_strange_P>5GeV                   :     950 from     1417 [ 67.04 %]     14 clones [ 1.45 %], purity: 99.30 %, hitEff: 89.54 %
UpstreamTrackChecker                   INFO   09_long_fromB                            :     267 from      377 [ 70.82 %]      2 clones [ 0.74 %], purity: 99.51 %, hitEff: 87.36 %
UpstreamTrackChecker                   INFO   09_long_fromD                            :    1813 from     2565 [ 70.68 %]     26 clones [ 1.41 %], purity: 99.59 %, hitEff: 88.37 %
UpstreamTrackChecker                   INFO   10_long_fromB_P>5GeV                     :     224 from      271 [ 82.66 %]      1 clones [ 0.44 %], purity: 99.58 %, hitEff: 87.71 %
UpstreamTrackChecker                   INFO   10_long_fromD_P>5GeV                     :    1559 from     1882 [ 82.84 %]     20 clones [ 1.27 %], purity: 99.66 %, hitEff: 88.98 %
UpstreamTrackChecker                   INFO   11_long_electrons                        :     547 from     2842 [ 19.25 %]     19 clones [ 3.36 %], purity: 98.20 %, hitEff: 86.24 %
UpstreamTrackChecker                   INFO   12_long_fromB_electrons                  :      13 from       39 [ 33.33 %]      1 clones [ 7.14 %], purity: 97.71 %, hitEff: 86.55 %
UpstreamTrackChecker                   INFO   13_long_fromB_electrons_P>5GeV           :      11 from       24 [ 45.83 %]      1 clones [ 8.33 %], purity: 97.33 %, hitEff: 84.31 %
UpstreamTrackChecker                   INFO   14_long_fromB_P>3GeV_Pt>0.5GeV           :     205 from      221 [ 92.76 %]      0 clones [ 0.00 %], purity: 99.57 %, hitEff: 86.64 %
UpstreamTrackChecker                   INFO   14_long_fromB_electrons_P>3GeV_Pt>0.5GeV :      10 from       15 [ 66.67 %]      1 clones [ 9.09 %], purity: 97.74 %, hitEff: 88.33 %
UpstreamTrackChecker                   INFO   14_long_fromD_P>3GeV_Pt>0.5GeV           :    1443 from     1538 [ 93.82 %]     18 clones [ 1.23 %], purity: 99.71 %, hitEff: 88.71 %
UpstreamTrackChecker                   INFO   14_long_strange_P>3GeV_Pt>0.5GeV         :     744 from      810 [ 91.85 %]     13 clones [ 1.72 %], purity: 99.45 %, hitEff: 89.05 %
UpstreamTrackChecker                   INFO   15_UT_long_fromB_P>3GeV_Pt>0.5GeV        :     205 from      221 [ 92.76 %]      0 clones [ 0.00 %], purity: 99.57 %, hitEff: 86.64 %

this MR

UpstreamTrackChecker                   INFO Results
UpstreamTrackChecker                   INFO **** Upstream                                  34229 tracks including           1892 ghosts [ 5.53 %], Event average  5.07 % ****
UpstreamTrackChecker                   INFO   01_velo                                  :   30354 from    65861 [ 46.09 %]    481 clones [ 1.56 %], purity: 99.46 %, hitEff: 87.13 %
UpstreamTrackChecker                   INFO   02_velo+UT                               :   30288 from    57215 [ 52.94 %]    481 clones [ 1.56 %], purity: 99.48 %, hitEff: 87.13 %
UpstreamTrackChecker                   INFO   03_velo+UT_P>5GeV                        :   21732 from    29341 [ 74.07 %]    356 clones [ 1.61 %], purity: 99.53 %, hitEff: 88.71 %
UpstreamTrackChecker                   INFO   04_velo+notLong                          :    6135 from    28074 [ 21.85 %]     88 clones [ 1.41 %], purity: 99.24 %, hitEff: 85.04 %
UpstreamTrackChecker                   INFO   05_velo+UT+notLong                       :    6078 from    19887 [ 30.56 %]     88 clones [ 1.43 %], purity: 99.33 %, hitEff: 85.01 %
UpstreamTrackChecker                   INFO   06_velo+UT+notLong_P>5GeV                :    3337 from     5322 [ 62.70 %]     61 clones [ 1.80 %], purity: 99.45 %, hitEff: 89.29 %
UpstreamTrackChecker                   INFO   07_long                                  :   24219 from    37787 [ 64.09 %]    393 clones [ 1.60 %], purity: 99.52 %, hitEff: 87.66 %
UpstreamTrackChecker                   INFO   07_long_strange                          :    1312 from     2576 [ 50.93 %]     21 clones [ 1.58 %], purity: 99.13 %, hitEff: 87.73 %
UpstreamTrackChecker                   INFO   08_long_P>5GeV                           :   18404 from    24385 [ 75.47 %]    295 clones [ 1.58 %], purity: 99.54 %, hitEff: 88.60 %
UpstreamTrackChecker                   INFO   08_long_strange_P>5GeV                   :     969 from     1417 [ 68.38 %]     13 clones [ 1.32 %], purity: 99.12 %, hitEff: 89.04 %
UpstreamTrackChecker                   INFO   09_long_fromB                            :     273 from      377 [ 72.41 %]      2 clones [ 0.73 %], purity: 99.42 %, hitEff: 87.15 %
UpstreamTrackChecker                   INFO   09_long_fromD                            :    1850 from     2565 [ 72.12 %]     26 clones [ 1.39 %], purity: 99.43 %, hitEff: 87.80 %
UpstreamTrackChecker                   INFO   10_long_fromB_P>5GeV                     :     229 from      271 [ 84.50 %]      1 clones [ 0.43 %], purity: 99.49 %, hitEff: 87.71 %
UpstreamTrackChecker                   INFO   10_long_fromD_P>5GeV                     :    1596 from     1882 [ 84.80 %]     20 clones [ 1.24 %], purity: 99.44 %, hitEff: 88.24 %
UpstreamTrackChecker                   INFO   11_long_electrons                        :     557 from     2842 [ 19.60 %]     19 clones [ 3.30 %], purity: 98.25 %, hitEff: 86.06 %
UpstreamTrackChecker                   INFO   12_long_fromB_electrons                  :      13 from       39 [ 33.33 %]      1 clones [ 7.14 %], purity: 97.82 %, hitEff: 90.12 %
UpstreamTrackChecker                   INFO   13_long_fromB_electrons_P>5GeV           :      11 from       24 [ 45.83 %]      1 clones [ 8.33 %], purity: 97.46 %, hitEff: 88.47 %
UpstreamTrackChecker                   INFO   14_long_fromB_P>3GeV_Pt>0.5GeV           :     211 from      221 [ 95.48 %]      0 clones [ 0.00 %], purity: 99.54 %, hitEff: 86.82 %
UpstreamTrackChecker                   INFO   14_long_fromB_electrons_P>3GeV_Pt>0.5GeV :      10 from       15 [ 66.67 %]      1 clones [ 9.09 %], purity: 97.88 %, hitEff: 92.88 %
UpstreamTrackChecker                   INFO   14_long_fromD_P>3GeV_Pt>0.5GeV           :    1474 from     1538 [ 95.84 %]     18 clones [ 1.21 %], purity: 99.52 %, hitEff: 88.06 %
UpstreamTrackChecker                   INFO   14_long_strange_P>3GeV_Pt>0.5GeV         :     755 from      810 [ 93.21 %]     13 clones [ 1.69 %], purity: 99.25 %, hitEff: 88.59 %
UpstreamTrackChecker                   INFO   15_UT_long_fromB_P>3GeV_Pt>0.5GeV        :     211 from      221 [ 95.48 %]      0 clones [ 0.00 %], purity: 99.54 %, hitEff: 86.82 %

Track resolution checks have been carried out as well. There are no significant changes:
current master

TrackResChecker                        INFO      ************************************
TrackResChecker                        INFO ALL/x pull     :  mean =  0.008 +/- 0.007, RMS = 0.981 +/- 0.007
TrackResChecker                        INFO ALL/y pull     :  mean =  0.006 +/- 0.007, RMS = 0.973 +/- 0.006
TrackResChecker                        INFO ALL/tx pull    :  mean =  -0.010 +/- 0.007, RMS = 1.023 +/- 0.007
TrackResChecker                        INFO ALL/ty pull    :  mean =  -0.006 +/- 0.007, RMS = 0.977 +/- 0.007
TrackResChecker                        INFO ALL/p pull     :  mean =  0.006 +/- 0.001, RMS = 0.147 +/- 0.007
TrackResChecker                        INFO ALL/probChi2   :  mean =  0.537 +/- 0.002, RMS = 0.320 +/- 0.001
TrackResChecker                        INFO ALL/x resolution / mm:  RMS =  43.809 +/- 0.551 micron
TrackResChecker                        INFO ALL/y resolution / mm:  RMS =  44.321 +/- 0.588 micron
TrackResChecker                        INFO ALL/dp/p:  mean =  -0.0009 +/- 0.0001, RMS =  0.0083 +/- 0.0001

this MR

TrackResChecker                        INFO      ************************************
TrackResChecker                        INFO ALL/x pull     :  mean =  0.007 +/- 0.007, RMS = 0.982 +/- 0.006
TrackResChecker                        INFO ALL/y pull     :  mean =  0.006 +/- 0.007, RMS = 0.976 +/- 0.006
TrackResChecker                        INFO ALL/tx pull    :  mean =  -0.009 +/- 0.007, RMS = 1.024 +/- 0.007
TrackResChecker                        INFO ALL/ty pull    :  mean =  -0.005 +/- 0.007, RMS = 0.980 +/- 0.007
TrackResChecker                        INFO ALL/p pull     :  mean =  0.006 +/- 0.001, RMS = 0.148 +/- 0.007
TrackResChecker                        INFO ALL/probChi2   :  mean =  0.537 +/- 0.002, RMS = 0.320 +/- 0.001
TrackResChecker                        INFO ALL/x resolution / mm:  RMS =  43.773 +/- 0.545 micron
TrackResChecker                        INFO ALL/y resolution / mm:  RMS =  44.602 +/- 0.586 micron
TrackResChecker                        INFO ALL/dp/p:  mean =  -0.0009 +/- 0.0001, RMS =  0.0083 +/- 0.0001

Throuput tests

master https://mattermost.web.cern.ch/lhcb/pl/qfcgykko47fz5fr4bt3rawrpge

Throughput of branch master (98884cfe), sequence hlt1_pp_scifi_v6 over dataset upgrade-magdown-sim10-up08-30000000-digi_01:
NVIDIA A10            │██████████████████████████████████████████████   155.09 kHz (1.00x)
GeForce RTX 3080      │███████████████████████████████████████████      145.58 kHz (1.00x)
GeForce RTX 3090      │███████████████████████████████████████████      145.52 kHz (1.00x)
A40                   │███████████████████████████████████████████      144.50 kHz (0.98x)
RTX A6000             │██████████████████████████████████████████       140.33 kHz (0.98x)
Quadro RTX 6000       │███████████████████████████████████████          132.44 kHz (1.00x)
MI100                 │███████████████████████████████████████          131.09 kHz (0.99x)
GeForce RTX 2080 Ti   │██████████████████████████████████████           128.51 kHz (0.99x)
Tesla V100-PCIE-32GB  │████████████████████████████████████             122.64 kHz (1.00x)
AMD EPYC 7502 32-Core │██████                                           20.81 kHz (0.99x)
Intel Xeon E5-2630 v4 │█                                                4.37 kHz (1.01x)
                      ┼──┴──┼──┴──┼──┴──┼──┴──┼──┴──┼──┴──┼──┴──┼──┴──┼
                      0     20    40    60    80   100   120   140   160  

this MR https://mattermost.web.cern.ch/lhcb/pl/63e1nqwd9bb99kymumyx3domjo

Throughput of branch mstahl_ut_cluster (bea5d981), sequence hlt1_pp_scifi_v6 over dataset upgrade-magdown-sim10-up08-30000000-digi_01:
NVIDIA A10            │████████████████████████████████     160.45 kHz (1.03x)
A40                   │█████████████████████████████        149.56 kHz (1.03x)
GeForce RTX 3090      │████████████████████████████         143.62 kHz (0.99x)
RTX A6000             │███████████████████████████          138.38 kHz (0.99x)
GeForce RTX 3080      │███████████████████████████          138.07 kHz (0.95x)
MI100                 │███████████████████████████          137.81 kHz (1.05x)
Quadro RTX 6000       │██████████████████████████           133.70 kHz (1.01x)
GeForce RTX 2080 Ti   │██████████████████████████           130.20 kHz (1.01x)
Tesla V100-PCIE-32GB  │████████████████████████             122.48 kHz (1.00x)
AMD EPYC 7502 32-Core │████                                 21.04 kHz (1.01x)
Intel Xeon E5-2630 v4 │▌                                    4.46 kHz (1.02x)
                      ┼─┴─┼─┴─┼─┴─┼─┴─┼─┴─┼─┴─┼─┴─┼─┴─┼─┴─┼
                      0   20  40  60  80 100 120 140 160 180 

[1] The tests have been carried out with all events (1046; only 100 for the plot) /eos/lhcb/grid/prod/lhcb/MC/Upgrade/XDIGI/00129676/0000/00129676_00000010_1.xdigi (Event type 25103102, Lc -> Lambda pi) from which 619 (66) events passed the GEC. Note that the GEC still takes single strip hits, not clusters.

Edited by Marian Stahl

Merge request reports