Skip to content

adding modifications for re-reco

Pedro Vieira De Castro Ferreira Da Silva requested to merge dev/rereco into master

General cleanup:

  • config files
  • snakemake configs
  • updating the step_*cfg.py files

Improvements to NANO:

  • throughput : fill tables with single loop plus two possibilities
    • use reserve and push_back instead of resize
    • declare columns as class variables and resize for first event + use edm::Span
  • energy and time as float (i really doubt we need a double)
  • change compression algorithm

Some numerology on the NANO using pedestal run 1726225194 (measured in vocms1002 using maxThreads=4 and 50k events)

With the default LZMA compression 9 (as in the TB) I get the following. The major improvement I'm convinced is because instead of doing two loops (one over DIGIs and one over RecHits) one moves to a single loop.

Version Throughput [ev/s]
Updated from !46 (merged) 113
Move to reserve+push_back + 1 loop 230
Move to 1/run+edm::Span + 1 loop 230

However moving from LZMA compression to ZSTD the gain in throughput is remarkable. There's an increase in the file size (50%) but I would say it's still ok for our purposes.

Compression Level Throughput [ev/s] Size [MB] (50k events)
LZMA 9 230 118
LZMA 5 248 118
ZSTD 9 351 155
ZSTD 5 1764 175

So in this merge request I propose to use for NANO:

  • edm::Span for the filling (tbc if it pays off with ZS events)
  • ZSTD + compression 5

Maybe @hqu has additional suggestions.

Merge request reports

Loading