Stream to multiple files

Description of developments

This MR builds on !800 (merged) to allow the saving to different output files depending on the HLT lines that fired - "streaming".

Streams do not have to save the same raw banks and have different routing bits for the use of the Online system. The streams themselves are configured in streams.py.

The top level options file then designates which lines go to which stream like so

def make_streams():
    d0_to_hh_lines = [builder() for builder in d0_to_hh.values()]
    b_to_jpsix_lines = [builder() for builder in b_to_jpsix.values()]
    linedict = dict(charm=d0_to_hh_lines, b2cc=b_to_jpsix_lines)
    return linedict

Note that if no stream is given (ie. make_streams does not return a dict) the lines will be assigned to the default stream.

Note that streaming is only possible for MDF output format currently and if streaming is requested with ROOT output an error is thrown.

The control flow now looks as follows

'CompositeCFNodes': [('moore',              ##Top level control node
                      'LAZY_AND',
                      ['hlt_decision',      ##No behaviour change - run all lines
                       'report_writers',    ##No behaviour change - return default writers
                       'stream_writers'],   ##New node
                      True),
                     ('stream_writers',
                      'NONLAZY_OR',
                      ['b2cc_moore',        ##`<stream>_moore` node
                       'charm_moore'],
                      False),
                     ('b2cc_moore',
                      'LAZY_AND',
                      ['LoKi__HDRFilter/LoKi__HDRFilter#1',                ##`HltDecReportsFilter`
                       'RawEventSimpleCombiner/RawEventSimpleCombiner#1',
                       'LHCb__MDFWriter/LHCb__MDFWriter#1'],
                      True),
                     ('charm_moore',
                      'LAZY_AND',
                      ['LoKi__HDRFilter/LoKi__HDRFilter#2',
                       'RawEventSimpleCombiner/RawEventSimpleCombiner#2',
                       'LHCb__MDFWriter/LHCb__MDFWriter#2'],
                      True)],

The new node stream_writers consists of <stream>_moore nodes that write to the output file stream + "_" + path based on the success of a HltDecReportsFilter which uses the lines of the stream.

Note that there are 2 instances of

• HltDecReportsFilter as the lines used in the decision are the lines of the stream
• RawEventSimpleCombiner as the Raw banks in each stream can be different (also the alg has force_location(stream + ROOT_RAW_EVENT_LOCATION for technical reasons. Downstream applications will read the RawEvent back to ROOT_RAW_EVENT_LOCATION)
• MDFWriter as the output file is now stream dependent.

Unit tests have been included. Hlt/Hlt2Conf/options/hlt2_test_streaming.py produces 2 streams; test_stream_A and test_stream_B with custom lines, RawBanks and routing bits. Hlt/Hlt2Conf/tests/options/hlt2_stream_[A, B]_check.py check the expected DecReports based on the lines of the strea, the expected RawBanks and routing bits.

Also in the MR is the unit test for the output of the sprucing, Hlt/Hlt2Conf/tests/options/spruce_check.py now checks as many things in the output of the Sprucing that I could think of including HLT2 and Spruce DecReports, protoparticles, RawBanks and the the saving of particles in the TES decay tree.

Developing notes

• Routing bits - have 2 banks

Using this snippet:

b2cc_hlt2_test_persistreco_fromfile.mdf

HltRoutingBits bank 0
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
46
HltRoutingBits bank 1
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
34

charm_hlt2_test_persistreco_fromfile.mdf

HltRoutingBits bank 0
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
46
HltRoutingBits bank 1
[0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2

The 34 and 2 are as expected but Im not sure where the 46 is coming from...

Have created #280 with idea on how to address

• Customise raw banks per stream

Test according to https://gitlab.cern.ch/lhcb/Moore/-/blob/7b92ac96d5df6752e8c3fc4f5d8eb8e5e246b603/Hlt/Moore/python/Moore/streams.py#L48

In charm_hlt2_test_persistreco_fromfile.mdf

>>> evt['/Event/DAQ/RawEvent'].banks(LHCb.RawBank.Rich).size()
350L

In test_hlt2_test_persistreco_fromfile.mdf

>>> evt['/Event/DAQ/RawEvent'].banks(LHCb.RawBank.Rich).size()
0L

• output_writers function for HLT1
• Unit tests to check raw bank contents of MDF
• Unit tests to check which lines -> which stream

Links

Work on DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/24.

Arising issues: #279 (closed), #280