Skip to content

[QEE] branch to collect contributions aiming for April12th. BW is Influenced by Allen!1372

Izaac Sanderswood requested to merge qee_upgrade_24 into 2024-patches

Sitting at/above 2024-patches with commit d86c8106

NB: This MR is influenced by Allen!1372, adding some Hlt1 Jet Lines. There are a set of Hlt2/Spruce inclusive jet lines (single and dijet) that filter on these new hlt1 jet lines, that cannot be evaluated centrally.
The jet team have locally estimated the impact of these lines on the exclusive sprucing footprint, +80MB/s (~35%) to the QEE bandwidths.
This was done before the compression changes in Moore (see: slide 4 ), so Hlt2 estimates are ~2.5 times too small to compare to current estimations.
Therefore for the table below we consider +200MB/s to Hlt2 and +80 MB/s at sprucing.

  • This is an approximation, a new estimation with the same compression changes has been requested from the jet team

BW Conclusions

This MR affects the QEE streams (estimated via local evaluations):

Process BW Change [MB/s]
Full +220 (~10%)
Turbo -70 (~10%)
Spruce +190 (~90%)

Where the fraction is w.r.t. QEE's BWs estimated locally.
NB: This includes the following assumptions:

  • New changes to no longer compress the DstData in Moore are included, so absolute Hlt2 bandwidths are ~2.5 times larger than physical bandwidths. See: slide 4
  • This includes the assumption that Allen!1372 will be merged in time.
Local BW Estimation

The local estimate below by me is not including the Hlt1 Jet line.

Evaluated with master (e264305d) on 06-Apr-24.

50k input events for Hlt2
(input events for Spruce_Concurrent = qee-full-stream retention on 50k events).

on qee_upgrade_24

Process Output Events Size (MB) Rate (kHz) BW (MB/s)
HLT2_Full 298 103 6.8 2,350
HLT2_Turbo 3827 31 87.3 710
Spruce_Concurrent 150 14 3.4 320

on master

Process Output Events Size (MB) Rate (kHz) BW (MB/s)
HLT2_Full 294 102 6.7 2,330
HLT2_Turbo 3635 34 82.9 780
Spruce_Concurrent 121 9.2 2.8 210


QEE MR Policy

MR Progress BW changes Short description
!3155 (merged) Merged None Renaming of Hlt2 line names
!3229 (merged) Merged -30 MB/s (7%) Turbo Loosen mu_pt cuts for DY but add pid_mu cuts to keep BW constant
!2929 (merged) Merged +7.3 MB/s (2%) Turbo TTrack Exotica search
!3152 (closed) Merged +80 MB/s (40%) Sprucing Asym Jet lines: W->c
!3240 (merged) Merged +50-100 MB/s (25-50%) Sprucing Use Hlt1 Jet lines to filter inclusive Hlt2 Jets. Influenced by Allen!1372 makes it difficult to estimate 🙁
!3242 (merged) Merged None Centralise the jet builder definitions.
!3261 (merged) Merged None, but ~10 MB/s (~2%) Turbo BW once 2024 Configuration is in Remove DiMuonNoIP trchi2 cut
!3248 (merged) Merged 30 MB/s (~5%) Turbo BW Large increase in eff for majo2leplep lines

NB: These estimates were evaluated before the compression changes

These MRs did not make it for the deadline.
MR Progress BW changes Short description
!3204 Draft/WIP Dep. tbc Dependency on Rec!3826. Add potential for a JetID cut
!3186 Postponed tbc Reduce Jet spruce throughput . Postponed

Most recent ci-test. !3151 (comment 7846187)


  • local evaluations over 50k.
  • New evaluation of the Hlt1 Jet line's impact. Been requested.
Edited by Luke Grazette

Merge request reports