Sprucing crashes on DST produced by Hlt2 on minbias MC

I have a new problem with running B&Q sprucing on dst files that I produced by running Hlt2 on upgrade_minbias_hlt1_filtered MC. The script crashes, and from the log (see below) it seems that it's because of default_Spruce_Packer (see part of log below). And as it seems it crashes when faces first J/psi candidate.

EventSelector                       SUCCESS Reading Event record 1. Record number within stream 1: 1
HltANNSvc                           WARNING key 0xae08f856 has an explicitly configured overrule -- using that...
default_Spruce_Packer                 ERROR auto LHCb::{anonymous}::Encoder::id(const string&) const : could not locate packedobjectlocation /Event/Spruce/HLT2/MuonPIDV2ToMuonTracks_Long_5cc1c2af/OutputMuonTrac\
ks (configured prefix=/Event/Spruce/) in table for key 0xae08f856
default_Spruce_Packer                 ERROR Maximum number of errors ( 'ErrorMax':1) reached.
HLTControlFlowMgr                     FATAL Event failed in Node HltPackedBufferWriter/default_Spruce_PackedBufferWriter : Error in algorithm execute
default_Spruce_Packer                 ERROR auto LHCb::{anonymous}::Encoder::id(const string&) const : could not locate packedobjectlocation /Event/Spruce/HLT2/MuonPIDV2ToMuonTracks_Long_5cc1c2af/OutputMuonTrac\
ks (configured prefix=/Event/Spruce/) in table for key 0xae08f856
default_Spruce_Packer                 ERROR Maximum number of errors ( 'ErrorMax':1) reached.
HLTControlFlowMgr                     FATAL Event failed in Node CombineRawBankViewsToRawEvent/EvtSize_default : Error in algorithm execute
HLTControlFlowMgr                     FATAL *** Event 7 on slot 0 failed! ***
HLTControlFlowMgr                      INFO Dumping Alg Exec State for slot 0:
  [slot: 0, incident: AlgFail]:

  +                               bandq_detached_pions_73d81c22  e: d f: 0 sc: Filter decision: FAILED
  +                      bandq_detached_Etac1S2SToHHHH_47dc772a  e: d f: 0 sc: Filter decision: FAILED
...
  • @sesen pointed out that "Culprit seems to be (see below). Muon PIDs are made from tracks that are not in default track container. So the issue is in Hlt2."
MuonPIDUnpacker_6ec2048e              DEBUG Unpacked linked location  /Event/HLT2/MuonPIDV2ToMuonTracks_Long_5cc1c2af/OutputMuonTracks

===== technical details:

I run on my local stack which was updated to master & recompiled around april 18 (with "x86_64_v2-centos7-gcc11+detdesc-opt") same result happens with running lb-run -c x86_64_v2-centos7-gcc11+detdesc-opt --nightly lhcb-master/latest Moore/master gaudirun.py <myopts.py>.

options file can be found in /afs/cern.ch/user/i/ipolyako/public/for_DPA_WP1:

  • hlt2_bandq_lines.py
  • spruce_opt.py

Full logs for both steps (with options.output_level=2) can be found in same folder:

  • hlt2_bandq_lines_1.log_nightly(_DEBUG)
  • spruce_opt_test.log_nightly(_DEBUG)
Edited by Ivan Polyakov