Skip to content

Enforce use of an encoding/decoding key (TCK) in all Hlt encoders/decoders, do not use online-reserved bits in SourceID, remove explicit mention of 'packed' locations, enable 'stable' persist reco locations

Gerhard Raven requested to merge streamline-hltdaq into master

Closes #194 (closed), #220 (closed), #221 (closed), #225 (closed), #70 (closed) also see #202

  • replace HltRawBankDecoderBase base class with a thread-safe service and standalone utility funtions, and update decoder classes accordingly
  • replace use of non-thread-safe Error and Warning with MsgCounter
  • remove support of partially decoding the run1-style combined Hlt1+Hlt2 rawbanks
  • put code in LHCb::Hlt::DAQ namespace
  • use IIndexedANNSvc for both encoding and decoding to get various int <-> string mappings ** note: this includes the ExecutionReportsWriter in HLTScheduler!!!
  • add GitANNSvc as new implementations of IIndexedANNSvc, and always push the relevant encoding table to the underlying git repository if not already present, using the git SHA1SUM to derive the encoding key, which is used as the label to look up the encoding table.
  • GitANNSvc uses a 'stack' of git repositories, and will fall-back to the 'next' repo in case the requested oid cannot be found
  • add the above encoding / decoding 'key' to PackedBuffer, and make sure it propagates when merging/splitting buffers. Add property to overrule the key obtained from the data (and add warnings if non-zero key in data is overruled, and if zero key is not overruled)
  • do not use the ANNSvc configurable as global variable written and read from a json file to record which TES locations + types are available on the output, and thus could (should) be configured in the read back application. Instead use a dedicated configuration-time generated json 'manifest' instead.
  • no longer split the 'large' amalgamated buffer which lives in the DstData RawBank into smaller individual buffers for each container which each individually go in the TES at the 'packed' locations. Instead, transform the DstData raw bank(s) into a map of { locationID -> individual buffer } for all buffers present inside the amalgamated buffer, and have all unpackers use this single object as input, directly producing their output from it -- as a result, there are no 'packed' locations anymore... so also remove mention of those from the encoding table.
  • enable the possibility to the explicitly specify 'stable' persist reco locations

Changes to the raw data bank format, and increment in the versions of Hlt{Sel,Dec}Reports and DstData

  • HltSelReports, HltDecReports, DstData: re-define the sourceID in the header to avoid using the 5 most significant now-reserved bits
  • HltDecReports: add the encoding key explicitly (in addition to the TCK), and verify that it reproduces the required int->string mapping
  • DstData: add the compression and encoding keys to the payload, make the embedded locationIDs refer to the original (i.e. unpacked) TES locations
  • adapt the decoding to deal with both old/new version

See also slides at https://indico.cern.ch/event/1179084/

Must be applied in conjunction with Allen!911 (merged) DaVinci!733 (merged) MooreAnalysis!91 (merged) Moore!1529 (merged) MooreOnline!142 (merged)

Edited by Sevda Esen

Merge request reports