Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • Moore Moore
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 216
    • Issues 216
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Jira
    • Jira
  • Merge requests 93
    • Merge requests 93
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • LHCbLHCb
  • MooreMoore
  • Merge requests
  • !1137

Use rawbank views for sprucing

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Nicole Skidmore requested to merge views_sprucing into master Nov 02, 2021
  • Overview 46
  • Commits 20
  • Pipelines 25
  • Changes 43

To go with DaVinci!618 (merged), MooreAnalysis!65 (merged), LHCb!3303 (merged)

Make Sprucing use RawBank::View to access persistreco objects rawbanks from previous passes

Motivation

This is a move towards the extensive use of "views" which will allow the Sprucing AND HLT2 to customise which rawbanks get persisted in a NEW RawEvent without mutilating the input RawEvent by using eg. BankKillers

Sprucing will now have a new RawEvent "Event/{stream}" where "stream"=default if none given and will not rely on the "DAQ/RawEvent"

This functionality does not exist for HLT2 yet either, customisation of which raw banks are output only appears to work when you use Brunel MC input with input_raw_format = 4.3. Running HLT1->HLT2 where input_raw_format = 0.3 means everything is in /DAQ/RawEvent and you cannot customise which (detector for instance) rawbanks are persisted

https://gitlab.cern.ch/lhcb-dpa/project/-/issues/124

Whats changed

Hlt/Moore/python/Moore/lines.py

  • Creation of PassLine which inherits from SpruceLine for pass-through Tesla like functionality. Makes config.py cleaner
  • PassLine only takes hlt2_filter_code to allow streaming

Hlt/Moore/python/Moore/persistence/serialisation.py

  • HLT2 and Sprucing now need access to the DstData RawEvent and View respectively. By returning the HltPackedDataWriter object we can now specify which we want in config.py

Hlt/Moore/python/Moore/config.py

  • process is now either 'hlt{1,2}', 'spruce' or 'pass' as there is now a PassLine object. This enabled some simplifying of report_writers_node
  • Note a 'pass' process will also create a Spruce DecReport
  • For 'spruce' and 'pass' LHCb__UnpackRawEvent unpacks all possible rawbanks from previous passes to RawBank::Views. This alg is prepended to moore_children
  • new_hlt_banks['DstData'] now takes either the HltPackedDataWriter RawEvent or View for HLT2 and Sprucing respectively. See description of serialisation.py
  • Same for new_hlt_banks['HltDecReports'] and new_hlt_banks['SpruceDecReports'] which take the "OutputRawEvent" or "OutputView" of HltDecReportsWriter respectively
  • No need for bankkiller anymore :)
  • Changed logic of stream_writer so that the locations created during the job can be treated differently to the locations to persist from the previous pass - new_locations and persist_locations respectively.
  • Moved some configuration of these locations from moore_control_flow to stream_writer
  • Sprucing uses CombineRawBankViewsToRawEvent to put Views of new_locations and persist_locations into new RawEvent

Hlt/Moore/python/Moore/tcks.py

  • Now grabs Hlt1SelectionID avaliable if HLT2 is run with allen_hlt1=True

Created Sprucing and Streaming folders for qmt tests and options

  • Big tidy up and consolidation of some testing scripts
Edited Jan 06, 2022 by Nicole Skidmore
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: views_sprucing