Skip to content

Fix FPE scanning and undefined variable in check_log.py script

Teng Jian Khoo requested to merge khoo/athena:24.0-TrigValTools-FixFPEScan into 24.0

Fixes some recent CI failures due to undeclared ignoreDict in some tests.

Also corrects the FPE message recognition logic to extract full threads, for example in test_trigID_bphys concerning ATR-29548:

Found 61 fpe message(s) in log.RDOtoRDOTrigger:
20:01:28 FPEAuditor                                          4351     6 WARNING FPE OVERFLOW in [Execute] of [BmumuxComboHypo] on event 4818 1 0x2ba0912aaa50
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 0 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libAthenaAuditors.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 1 :
20:01:28   in library : /lib64/libc.so.6
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 2 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtCore.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 3 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtCore.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 4 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtCore.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 5 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtFitterLib.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 6 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtFitterLib.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 7 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrkVKalVrtFitterLib.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 8 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrigBphysHypo.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 9 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrigBphysHypo.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 10 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrigBphysHypo.so
20:01:28 FPEAuditor                                          4351     6    INFO FPE stacktrace 11 :
20:01:28   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/Athena/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libTrigBphysHypo.so
20:01:30 FPEAuditor                                          4351     6    INFO FPE stacktrace 12 :
20:01:30   in function : Gaudi::Algorithm::sysExecute(EventContext const&) (/build/atnight/localbuilds/nightlies/Athena/24.0/build/build/AthenaExternals/src/Gaudi/GaudiKernel/src/Lib/Algorithm.cpp:352)
20:01:30   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/AthenaExternals/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libGaudiKernel.so
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 13 :
20:01:31   in function : AlgTask::operator()() const (/build/atnight/localbuilds/nightlies/Athena/24.0/build/build/AthenaExternals/src/Gaudi/GaudiHive/src/AlgTask.h:81)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/AthenaExternals/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libGaudiHive.so
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 14 :
20:01:31   in function : tbb::internal::function_task<AlgTask>::execute() (/cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-2e3ca/x86_64-el9-gcc13-opt/include/tbb/task.h:1061)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/2024-06-09T2101/AthenaExternals/24.0.48/InstallArea/x86_64-el9-gcc13-opt/lib/libGaudiHive.so
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 15 :
20:01:31   in function : tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::process_bypass_loop(tbb::internal::context_guard_helper<false>&, tbb::task*, long) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/custom_scheduler.h:474)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 16 :
20:01:31   in function : tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/custom_scheduler.h:636)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 17 :
20:01:31   in function : tbb::internal::arena::process(tbb::internal::generic_scheduler&) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/arena.cpp:196)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 18 :
20:01:31   in function : tbb::internal::market::process(rml::job&) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/market.cpp:668)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 19 :
20:01:31   in function : int tbb::internal::__TBB_machine_atomic_load<int, 2>(int const volatile&) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../include/tbb/machine/gcc_generic.h:160)
20:01:31   included from : tbb::internal::machine_load_store<int, 4ul>::load_with_acquire(int const volatile&) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../include/tbb/machine/gcc_generic.h:175)
20:01:31   included from : int tbb::internal::__TBB_load_with_acquire<int>(int const volatile&) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../include/tbb/tbb_machine.h:710)
20:01:31   included from : tbb::internal::atomic_impl<tbb::internal::rml::private_worker::state_t>::operator tbb::internal::rml::private_worker::state_t() const volatile (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../include/tbb/atomic.h:310)
20:01:31   included from : tbb::internal::rml::private_worker::run() (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/private_server.cpp:264)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 20 :
20:01:31   in function : tbb::internal::rml::private_worker::thread_routine(void*) (/build/jenkins/workspace/lcg_release_pipeline/build/externals/tbb-2020_U2/src/tbb/2020_U2/build/linux_intel64_gcc_cc13.1.0_libc2.34_kernel4.18.0_release/../../src/tbb/private_server.cpp:221)
20:01:31   in library : /cvmfs/atlas-nightlies.cern.ch/repo/sw/24.0_Athena_x86_64-el9-gcc13-opt/sw/lcg/releases/LCG_104d_ATLAS_7/tbb/2020_U2/x86_64-el9-gcc13-opt/lib/libtbb.so.2
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 21 :
20:01:31   in library : /lib64/libc.so.6
20:01:31 FPEAuditor                                          4351     6    INFO FPE stacktrace 22 :
20:01:31   in function : __GI___clone (:0)
20:01:31   in library : /lib64/libc.so.6

FYI @martindl @carquin

Merge request reports