Skip to content

New separate RDOtoRDOTrigger MT transform skeleton without any RecExCommon includes

Rafal Bielski requested to merge rbielski/athena:rdo2rdotrig-mt into master

Main goal of this MR is to avoid any RecExCommon includes in RDOtoRDOTrigger transform running the Run-3 trigger (MT).

In addition to the previous configuration being patchy, error-prone and difficult to read, it was actually found recently that the RecExCommon_topOptions include was interfering with the trigger configuration and introducing incorrect configuration. One example was ATR-21707 and another was !31712 (comment 3594445). There is also another one uncovered by this MR and changing trigger counts. Namely, offline calo cell maker was scheduled and ran before the trigger algorithms and HLT calo cell maker. Removing this misconfiguration slightly changes electron, jet and MET counts (cc @fernando, @cjmeyer, @khoo, @jbossios, @jburr, @damazio, @joheinri).

This MR makes RDOtoRDOTrigger in MT simply a thin wrapper around runHLT_standalone which is the main trigger job options file used for all production jobs (P1 data taking and offline MC production). With this MR, RDOtoRDOTrigger becomes exactly equivalent to running:

athena.py --filesInput=RDO.pool.root -c "doWriteBS=False; doWriteRDOTrigger=True;" TriggerJobOpts/runHLT_standalone.py

This has a few very important implications:

  1. Trigger MC prod configuration becomes:
    • completely contained within Trigger code domain
    • identical to simple athena.py job with runHLT_standalone options
    • as close as possible to online configuration (steered from the same job opts file)
    • clean and free from dubious nested includes
  2. The test test_trigAna_RDOtoAOD_mt1_build.py runs the same job and, most importantly, produces identical counts as the test test_trig_mc_v1Dev_build.py. This should be always the case but wasn't until this MR. This was clearly a bug which was possibly never checked.
  3. Since proper RDO_TRIG output has to be provided now by runHLT_standalone, it means simple athena.py job (like all TriggerTest tests) also produces a fully valid RDO_TRIG file with all other objects. This can be used as input to offline reconstruction. Potentially very useful for testing subsets of the menu from RDO to HIST/PhysVal.

All this is achieved in the following way:

  1. Move all legacy (Run-2 trigger) configuration from skeleton.RDOtoRDOtrigger.py to skeleton.RDOtoRDOtriggerLegacy.py.
  2. Create new skeleton.RDOtoRDOtriggerMT.py which is only passing the input arguments to runHLT_standalone.py.
  3. Make skeleton.RDOtoRDOtrigger.py a trivial forwarding script which forwards to one of the two depending on whether --threads option is used (athenaMT or legacy).
  4. Clean up the legacy code a bit and enforce flake8 checks for the RDOtoRDOtrigger* files in RecJobTransforms.
  5. Update POOL output configuration in TriggerConfig.py to produce full RDO_TRIG output (all input objects plus the trigger objects).
    • Add also ComponentAccumulator configuration for RoIBResultToxAOD and add it in the POOL output configuration.
  6. Update AutoConfiguration to accept "OutputStreamRDO" as valid name for input RDO, as this is produced by OutputStreamCfg.

Notes for diff review:

  • Due to a change in MessageSvc format there is an extra space in the reference file in every line. Please disable showing whitespace changes in the diff page to see the relevant changes (count changes in the reference).
  • Please also note that skeleton.RDOtoRDOtriggerLegacy.py is 95% existing code, just moved to another file and cleaned up slightly. I don not intend to rewrite the legacy configuration because it is not relevant for Run 3 and will soon become obsolete.

FYI: @jpanduro, @okumura, @smh, @tbold (please tag others who should be aware)

Edited by Rafal Bielski

Merge request reports