Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • L LHCb
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 193
    • Issues 193
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
  • Jira
    • Jira
  • Merge requests 44
    • Merge requests 44
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • LHCbLHCb
  • LHCb
  • Merge requests
  • !3268

New persistency model for packing

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Sevda Esen requested to merge sevda-packers2 into master Sep 27, 2021
  • Overview 320
  • Commits 1
  • Pipelines 138
  • Changes 48

A significant re-write of persistency framework.

Main changes in c++ are:

  • New packers take unpacked objects and produce HltPackedDataBuffers (MC packers unchanged, old packers fully removed from configuration)
  • New HltPackedBufferWriter takes a list of DataBuffers and writes out the raw banks/event (This replaces HltPackedDataWriter)
  • New HltPackedBufferDecoder takes rawbanks/event and produces HltPackedDataBuffers to the requested locations
  • New unpackers take HltPackedDataBuffers and produce unpacked objects (MC unpackers unchanged, old unpackers are still used for old dsts.)
  • Un/PackParticlesAndVertices algorithms are removed. So objects in TES are not automatically packed/writen/decoded/unpacked.

Main changes in configuration:

  • Packers/Unpackers/Decoder/Writer are all functional so all input and output locations needed to be given [They are given as DataObjectRead(Write)Handles. See #180 for explanation.]
  • For packing/writing, locations are picked up from reco objects list and requested-line dependencies. For each location, object type in that location is also registered to ANNSvc json file. Only known object types in PyConf/object_types are packed.
  • For decoding/unpacking, we don't know the data dependencies at the moment. So we take all registered packed locations in requested stream. They are checked against registered object types to decide which unpacker to use.
  • PersistRecoConf is extended to all object types not only reconstruction objects. More than one location can be given for a type. reconstruction_objects list should be modified to benefit from this. Moore#248

To go with Moore!1085 (merged) MooreAnalysis!69 (merged) DaVinci!634 (merged) lhcb-datapkg/PRConfig!213 (merged)

Related issues: #151 Moore#354 (closed) lhcb-dpa/project#120

Edited Apr 12, 2022 by Rosen Matev
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: sevda-packers2