Draft: Allow a limited number of exceptions to be 'silently' handled during data unpacking
See Moore#838 and lhcb-dpa/prod-requests#76 (comment 8461101)
This MR updates the unpackers to allow a limited (<5) number of exceptions during the unpacking to be 'silently' handled, in that an error counter is incremented, the output data is emptied, but the event processing is allowed to continue. The idea being one rare problem event in handful of input files should not prevent the rest of that file being processed OK, if it can.
The 'if it can' part is implemented by only allowing a small number of exceptions to be handled this way. Any more, and the algorithm will rethrow it and thus allow the application to die.
With, this I was able to process LFN:/lhcb/data/2024/RAW/TURCAL/LHCb/COLLISION24/305687/305687_00220009_0003.raw, one of the problem files above, to the end with just two exceptions to be caught (only one corrupt event, seen by two different decoder instances).
<snip>
EventSelector SUCCESS Reading Event record 40001. Record number within stream 1: 40001
HltANNSvc WARNING key 0x864a4524 has an explicitly configured overrule -- using that...
HLTControlFlowMgr INFO Timing started at: 13:50:03
Unpack_Hlt2__Event_HLT2_Hlt2Trac... ERROR Exception during unpacking
Unpack_Hlt2__Event_HLT2_Hlt2Trac... ERROR Exception 'ByteBuffer(): requested a read beyond buffer size!' Instance 1/5 allowed
Unpack_Hlt2__Event_HLT2_Hlt2Trac... ERROR Exception during unpacking
Unpack_Hlt2__Event_HLT2_Hlt2Trac... ERROR Exception 'ByteBuffer(): requested a read beyond buffer size!' Instance 1/5 allowed
EventSelector SUCCESS Reading Event record 50001. Record number within stream 1: 50001
<snip>The protections also handle a number of other errors seen in the report above, basically anything that results in an std::exception being thrown during the unpacking.