Segfault from FastJetBuilder (fastjet) due to NaN data in input particles.
Update : New information from gdb directly shows the issue is with the jets
Thread 10 "eventLoop" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffe8a7fc640 (LWP 1078884)]
0x00007fffa1ff8e19 in fastjet::ClusterSequence::_tj_set_jetinfo (_jets_index=73, jet=0x7ffe3813d3e8, this=0x7ffe8a7fb0e0)
at /build/jenkins/workspace/lcg_release_pipeline/build/externals/fastjet-3.4.1/src/fastjet/3.4.1/src/ClusterSequence_TiledN2.cc:213
warning: 213 /build/jenkins/workspace/lcg_release_pipeline/build/externals/fastjet-3.4.1/src/fastjet/3.4.1/src/ClusterSequence_TiledN2.cc: No such file or directory
(gdb) where
#0 0x00007fffa1ff8e19 in fastjet::ClusterSequence::_tj_set_jetinfo (_jets_index=73, jet=0x7ffe3813d3e8, this=0x7ffe8a7fb0e0)
at /build/jenkins/workspace/lcg_release_pipeline/build/externals/fastjet-3.4.1/src/fastjet/3.4.1/src/ClusterSequence_TiledN2.cc:213
#1 fastjet::ClusterSequence::_faster_tiled_N2_cluster (this=0x7ffe8a7fb0e0)
at /build/jenkins/workspace/lcg_release_pipeline/build/externals/fastjet-3.4.1/src/fastjet/3.4.1/src/ClusterSequence_TiledN2.cc:532
#2 0x00007fff769c83cc in fastjet::ClusterSequence::ClusterSequence<fastjet::PseudoJet> (this=0x7ffe8a7fb0e0, pseudojets=std::vector of length 74, capacity 74 = {...},
jet_def_in=..., writeout_combinations=<optimized out>) at /cvmfs/lhcb.cern.ch/lib/lcg/releases/fastjet/3.4.1-5af57/x86_64-el9-gcc13-dbg/include/fastjet/ClusterSequence.hh:1028
#3 0x00007fff769ac919 in FastJetBuilder::makeFastJets (this=this@entry=0x83227d20, particles=..., geometry=...) at ../Phys/JetAccessories/src/FastJetBuilder.cpp:611
#4 0x00007fff769afa5b in FastJetBuilder::operator() (this=this@entry=0x83227d20, Particles=..., PrimaryVertices=..., geometry=...)
at ../Phys/JetAccessories/src/FastJetBuilder.cpp:490see discussion threads below.
Original description :
Crash observed in HLT64618 processing run 303125 with hlt2_pp_2024.
The debugger points to BanksToRawEvent::operator():
[cmarinbe@n8190601 ~]$ /group/hlt/hlt2/stack_RTA_2024_07_09_3/MooreOnline/gdb /group/hlt/hlt2/stack_RTA_2024_07_09_3/MooreOnline/build.x86_64_v3-el9-gcc13-opt+g/Online/Converters/libOnlineConverters.so
(gdb) list*(_ZNK6Online15BanksToRawEventclERKSt6vectorISt4pairIPKN4LHCb7RawBankEPKvESaIS9_EE+0x7e)
(...)
0x958de is in Online::BanksToRawEvent::operator()(std::vector<std::pair<LHCb::RawBank const*, void const*>, std::allocator<std::pair<LHCb::RawBank const*, void const*> > > const&) const (/cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/include/Event/RawBank.h:240).
235 int totalSize() const {
236 typedef unsigned int T;
237 return m_length % sizeof( T ) == 0 ? m_length : ( m_length / sizeof( T ) + 1 ) * sizeof( T );
238 }
239 /// Return bankType of this bank
240 BankType type() const { return BankType( int( m_type ) ); }
241
242 /// Set the bank type
243 RawBank& setType( BankType val ) {
244 m_type = (unsigned char)( char( val ) & 0xFF );Files where one should be able to reproduce the crash are available under: /hlt2/errors/LHCb2/0000303125/
Full stack trace:
Aug 8, 2024 @ 05:19:48.002 Writer WARN Opened /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051948-001.0005.mep
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 00 --> 0x7fffa8771da0
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 05 --> 0x7fffd485b500
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 11 --> 0x7ffff7739802
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 04 --> 0x7fffabab3389
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 02 --> 0x7ffff76eed90
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 03 --> 0x7fffabaa68de
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 12 --> 0x7ffff76d9450
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 09 --> 0x7fffd1e46ea8
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (ExitSignalHandler) ---------------------- Backtrace ----------------------
Aug 8, 2024 @ 05:19:47.954 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x7fff4a9ff724 Code:00000001
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 10 --> 0x7fffd1e490ee
Aug 8, 2024 @ 05:19:47.954 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x7fff5fd3ba8d Code:00000001
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 01 --> 0x7fffa87766c3
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 08 --> 0x7fffa7098eba
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 06 --> 0x7fffa70a6243
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' Number of elements in backtrace: 13
Aug 8, 2024 @ 05:19:47.954 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 07 --> 0x7fffa70986e8
Aug 8, 2024 @ 05:19:47.953 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x7fff5fd294b5 Code:00000001
Aug 8, 2024 @ 05:19:47.953 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (ExitSignalHandler) ---------------------- Backtrace ----------------------
Aug 8, 2024 @ 05:19:47.953 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x7fff5fc2afc4 Code:00000001
Aug 8, 2024 @ 05:19:47.953 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' Number of elements in backtrace: 13
Aug 8, 2024 @ 05:19:47.950 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (ExitSignalHandler) ---------------------- Backtrace ----------------------
Aug 8, 2024 @ 05:19:47.950 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' Number of elements in backtrace: 15
Aug 8, 2024 @ 05:19:47.949 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x7fff31fa0c5c Code:00000001
Aug 8, 2024 @ 05:19:47.901 Writer WARN Closed /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-898.0004.mep after 0 MB [event-limit].
Aug 8, 2024 @ 05:19:47.900 Writer WARN Opened /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-898.0004.mep
Aug 8, 2024 @ 05:19:47.898 Writer WARN Opened /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-897.0003.mep
Aug 8, 2024 @ 05:19:47.898 Writer WARN Closed /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-897.0003.mep after 0 MB [event-limit].
Aug 8, 2024 @ 05:19:47.844 Writer WARN save_mdf: Encountered invalid MDF header: (-469755108 419423676 -369091651). Skip 211564 bytes of data.
Aug 8, 2024 @ 05:19:47.796 Writer WARN Closed /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-791.0002.mep after 0 MB [event-limit].
Aug 8, 2024 @ 05:19:47.796 Writer WARN Opened /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-791.0002.mep
Aug 8, 2024 @ 05:19:47.791 Writer WARN Opened /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-779.0001.mep
Aug 8, 2024 @ 05:19:47.791 Writer WARN Closed /hlt2/errors/LHCb2/0000303125/Error_0000303125_HLT64618_20240808-051947-779.0001.mep after 0 MB [event-limit].
Aug 8, 2024 @ 05:19:47.781 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.8 accident event: 0x7f9012937000 len: 15095284 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.772 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.10 accident event: 0x7f901c0f7000 len: 15263280 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.764 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.14 accident event: 0x7f900eeb7000 len: 14326056 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.756 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.15 accident event: 0x7f900fc77000 len: 14954560 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.748 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.16 accident event: 0x7f90235b7000 len: 14461080 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.740 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.17 accident event: 0x7f901b277000 len: 14957896 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.732 MEPManager ERROR [Events_LHCb2] Client LHCb2_HLT64618_HLT2_0.19 accident event: 0x7f9033ff7000 len: 14773868 typ: 2 mask: [FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF]
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 15 --> 0x7ffff76d9450
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 10 --> 0x7fffa70986e8
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 04 --> 0x7fff9002193d
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 02 --> 0x7ffff76eed90
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 05 --> 0x7fff8ffeb9c4
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 00 --> 0x7fffa8771da0
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 01 --> 0x7fffa87766c3
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 06 --> 0x7fff8ffed275
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 03 --> 0x7fff8feb11fd
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 11 --> 0x7fffa7098eba
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 08 --> 0x7fffd485b500
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 12 --> 0x7fffd1e46ea8
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 13 --> 0x7fffd1e490ee
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 14 --> 0x7ffff7739802
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 07 --> 0x7fff90033756
Aug 8, 2024 @ 05:19:47.731 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) 09 --> 0x7fffa70a6243
Aug 8, 2024 @ 05:19:47.729 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' (ExitSignalHandler) ---------------------- Backtrace ----------------------
Aug 8, 2024 @ 05:19:47.729 [INFO] Process: 'LHCb2_HLT64618_HLT2_0' Number of elements in backtrace: 16
Aug 8, 2024 @ 05:19:47.696 [FATAL] Process: 'LHCb2_HLT64618_HLT2_0' (SignalHandler) RTL:Handled signal: 11 [SIGSEGV] Old action:0x7fffef515ae0 Mem:0x69 Code:00000001
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x54d90)[0x7ffff76eed90]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v38r1/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libGaudiKernel.so(_ZN5Gaudi9Algorithm10sysExecuteERK12EventContext+0x180)[0x7fffd485b500]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2021.10.0-2a247/x86_64-el9-gcc13-opt/lib64/libtbb.so.12(+0x22ea8)[0x7fffd1e46ea8]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libGaudiOnlineComp.so(_ZNK6Online14EventProcessor7executeERK12EventContext+0x2f)[0x7fffa8a1024f]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(_ZNK10AlgWrapper7executeER12EventContextN3gsl4spanIN4LHCb10Interfaces23ISchedulerConfiguration5State8AlgStateELm18446744073709551615EEE+0x53)[0x7fffa70a6243]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Rec/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libJetAccessories.so(_ZNK14FastJetBuilderclERKN5Gaudi11NamedRange_ISt6vectorIPKN4LHCb8ParticleESaIS6_EEN9__gnu_cxx17__normal_iteratorIPKS6_S8_EEEERK14KeyedContainerINS3_9RecVertexEN10Containers18KeyedObjectManagerINSJ_7hashmapEEEERKNS3_8Detector12DeIOVElementINSQ_6detail11DeIOVObjectEEE+0x955)[0x7fff8ffed275]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libEventHandling.so.7.27.0.0(+0x1448a)[0x7fffa883d48a]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Rec/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libJetAccessories.so(_ZNK5Gaudi10Functional7details11TransformerIF14KeyedContainerIN4LHCb8ParticleEN10Containers18KeyedObjectManagerINS6_7hashmapEEEERKNS_11NamedRange_ISt6vectorIPKS5_SaISE_EEN9__gnu_cxx17__normal_iteratorIPKSE_SG_EEEERKS3_INS4_9RecVertexES9_ERKNS4_8Detector12DeIOVElementINST_6detail11DeIOVObjectEEEENS0_6Traits4use_IJNS4_3Det8LbDD4hep21useConditionHandleForIJSX_EEENS11_11BaseClass_tINS14_23ConditionAccessorHolderI10FixTESPathINS_9AlgorithmEEEEEEEEELb0EE7executeERK12EventContext+0x106)[0x7fff90033756]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(+0x141da0)[0x7fffa8771da0]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x54d90)[0x7ffff76eed90]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x54d90)[0x7ffff76eed90]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(+0xee6e8)[0x7fffa70986e8]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(+0x141da0)[0x7fffa8771da0]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libEventHandling.so.7.27.0.0(+0x15479)[0x7fffa883e479]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/MooreOnline/build.x86_64_v3-el9-gcc13-opt+g/Online/Converters/libOnlineConverters.so(_ZNK6Online15BanksToRawEventclERKSt6vectorISt4pairIPKN4LHCb7RawBankEPKvESaIS9_EE+0x7e)[0x7fffabaa68de]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v38r1/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libGaudiKernel.so(_ZN5Gaudi9Algorithm10sysExecuteERK12EventContext+0x180)[0x7fffd485b500]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(_ZN3RTL17ExitSignalHandler7handlerEiP9siginfo_tPv+0x2b3)[0x7fffa87766c3]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x9f802)[0x7ffff7739802]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2021.10.0-2a247/x86_64-el9-gcc13-opt/lib64/libtbb.so.12(+0x250ee)[0x7fffd1e490ee]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x3f450)[0x7ffff76d9450]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/MooreOnline/build.x86_64_v3-el9-gcc13-opt+g/Online/Converters/libOnlineConverters.so(_ZNK5Gaudi10Functional7details11TransformerIFN4LHCb8RawEventERKSt6vectorISt4pairIPKNS3_7RawBankEPKvESaISC_EEENS0_6Traits4use_IJEEELb0EE7executeERK12EventContext+0x59)[0x7fffabab3389]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(+0x141da0)[0x7fffa8771da0]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Rec/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libJetAccessories.so(_ZNK14FastJetBuilder12makeFastJetsEN3gsl4spanIKPKN4LHCb8ParticleELm18446744073709551615EEERKNS2_8Detector12DeIOVElementINS8_6detail11DeIOVObjectEEE+0x2a4)[0x7fff8ffeb9c4]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(+0xeeeba)[0x7fffa7098eba]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v38r1/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libGaudiKernel.so(_ZN5Gaudi9Algorithm10sysExecuteERK12EventContext+0x180)[0x7fffd485b500]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(+0xeeeba)[0x7fffa7098eba]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2021.10.0-2a247/x86_64-el9-gcc13-opt/lib64/libtbb.so.12(+0x22ea8)[0x7fffd1e46ea8]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(+0x141da0)[0x7fffa8771da0]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x54d90)[0x7ffff76eed90]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(+0xee6e8)[0x7fffa70986e8]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x3f450)[0x7ffff76d9450]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(_ZN3RTL17ExitSignalHandler7handlerEiP9siginfo_tPv+0x2b3)[0x7fffa87766c3]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2021.10.0-2a247/x86_64-el9-gcc13-opt/lib64/libtbb.so.12(+0x250ee)[0x7fffd1e490ee]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libGaudiOnlineComp.so(_ZNK6Online9OutputAlg7processERK12EventContext+0x15d)[0x7fffa8a80acd]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(_ZNK10AlgWrapper7executeER12EventContextN3gsl4spanIN4LHCb10Interfaces23ISchedulerConfiguration5State8AlgStateELm18446744073709551615EEE+0x53)[0x7fffa70a6243]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lcg/releases/fastjet/3.4.1-5af57/x86_64-el9-gcc13-opt/lib/libfastjet.so.0(_ZN7fastjet15ClusterSequence24_faster_tiled_N2_clusterEv+0x56d)[0x7fff8feb11fd]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(+0xee6e8)[0x7fffa70986e8]
Aug 8, 2024 @ 05:19:47.000 /lib64/libc.so.6(+0x9f802)[0x7ffff7739802]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(_ZN3RTL17ExitSignalHandler7handlerEiP9siginfo_tPv+0x2b3)[0x7fffa87766c3]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Online/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libOnlineBase.so.7.27.0.0(_ZN3RTL17ExitSignalHandler7handlerEiP9siginfo_tPv+0x2b3)[0x7fffa87766c3]
Aug 8, 2024 @ 05:19:47.000 /group/hlt/hlt2/stack_RTA_2024_07_09_3/Rec/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libJetAccessories.so(_ZN7fastjet15ClusterSequenceC1INS_9PseudoJetEEERKSt6vectorIT_SaIS4_EERKNS_13JetDefinitionERKb+0x46d)[0x7fff9002193d]
Aug 8, 2024 @ 05:19:47.000 /cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v55r11/InstallArea/x86_64_v3-el9-gcc13-opt+g/lib/libHLTScheduler.so(_ZNK10AlgWrapper7executeER12EventContextN3gsl4spanIN4LHCb10Interfaces23ISchedulerConfiguration5State8AlgStateELm18446744073709551615EEE+0x53)[0x7fffa70a6243]Edited by Christopher Rob Jones