Skip to content

Use rawbank views for sprucing

Nicole Skidmore requested to merge views_sprucing into master

To go with DaVinci!618 (merged), MooreAnalysis!65 (merged), LHCb!3303 (merged)

Make Sprucing use RawBank::View to access persistreco objects rawbanks from previous passes

Motivation

This is a move towards the extensive use of "views" which will allow the Sprucing AND HLT2 to customise which rawbanks get persisted in a NEW RawEvent without mutilating the input RawEvent by using eg. BankKillers

Sprucing will now have a new RawEvent "Event/{stream}" where "stream"=default if none given and will not rely on the "DAQ/RawEvent"

This functionality does not exist for HLT2 yet either, customisation of which raw banks are output only appears to work when you use Brunel MC input with input_raw_format = 4.3. Running HLT1->HLT2 where input_raw_format = 0.3 means everything is in /DAQ/RawEvent and you cannot customise which (detector for instance) rawbanks are persisted

https://gitlab.cern.ch/lhcb-dpa/project/-/issues/124

Whats changed

Hlt/Moore/python/Moore/lines.py

  • Creation of PassLine which inherits from SpruceLine for pass-through Tesla like functionality. Makes config.py cleaner
  • PassLine only takes hlt2_filter_code to allow streaming

Hlt/Moore/python/Moore/persistence/serialisation.py

  • HLT2 and Sprucing now need access to the DstData RawEvent and View respectively. By returning the HltPackedDataWriter object we can now specify which we want in config.py

Hlt/Moore/python/Moore/config.py

  • process is now either 'hlt{1,2}', 'spruce' or 'pass' as there is now a PassLine object. This enabled some simplifying of report_writers_node
  • Note a 'pass' process will also create a Spruce DecReport
  • For 'spruce' and 'pass' LHCb__UnpackRawEvent unpacks all possible rawbanks from previous passes to RawBank::Views. This alg is prepended to moore_children
  • new_hlt_banks['DstData'] now takes either the HltPackedDataWriter RawEvent or View for HLT2 and Sprucing respectively. See description of serialisation.py
  • Same for new_hlt_banks['HltDecReports'] and new_hlt_banks['SpruceDecReports'] which take the "OutputRawEvent" or "OutputView" of HltDecReportsWriter respectively
  • No need for bankkiller anymore :)
  • Changed logic of stream_writer so that the locations created during the job can be treated differently to the locations to persist from the previous pass - new_locations and persist_locations respectively.
  • Moved some configuration of these locations from moore_control_flow to stream_writer
  • Sprucing uses CombineRawBankViewsToRawEvent to put Views of new_locations and persist_locations into new RawEvent

Hlt/Moore/python/Moore/tcks.py

  • Now grabs Hlt1SelectionID avaliable if HLT2 is run with allen_hlt1=True

Created Sprucing and Streaming folders for qmt tests and options

  • Big tidy up and consolidation of some testing scripts
Edited by Nicole Skidmore

Merge request reports