Segmentation violation error when running unfold command with systematics
When running the unfold command at some point I get the below error. I got this error when I ran the command only with the -s option.
Looks like the error happens after this:
Error in <TFile::mkdir>: An object with name AbsoluteStatDown exists already$ unfold U1_getUnfHist/JetHT_UL18_CHS_RunABCD.root U1_getUnfHist/Pythia_FlatUL18_CHS.root U2_unfold/data_unf_MC_UL18_CHS.root -s
Warning: you are overwriting "U2_unfold/data_unf_MC_UL18_CHS.root"
unfold 0/1 start
451 Opening input files
463 Preparing output file
167 folding
85 Calculating condition
0 0.720625 1
1 0.718145 1.00345
2 0.712797 1.01098
3 0.711154 1.01332
4 0.710221 1.01465
5 0.707065 1.01918
...
...
...
Info in <TUnfoldDensityV17::TUnfold>: *NOT* unfolding bin #143 (xaxis:gen[142,144])
Info in <TUnfoldDensityV17::TUnfold>: *NOT* unfolding bin #144 (xaxis:gen[144,144])
Info in <TUnfoldDensityV17::TUnfold>: *NOT* unfolding bin #145 (xaxis:gen[ofl])
Warning in <TUnfoldDensityV17::SetInput>: 54/288 input bins have zero error, and are ignored.
651 unfold status = 54
Error in <TNetXNGFile::mkdir>: An object with name AbsoluteStatDown exists already
*** Break *** segmentation violation
===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
Thread 6 (Thread 0x7fab937fb640 (LWP 1169726) "unfold"):
#0 0x00007fabb48868ba in __futex_abstimed_wait_common () from /lib64/libc.so.6
#1 0x00007fabb4891d68 in __new_sem_wait_slow64.constprop.0 () from /lib64/libc.so.6
#2 0x00007fab95983e56 in XrdSysSemaphore::Wait (this=0x3bc3ec0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdSys/XrdSysPthread.hh:509
#3 XrdCl::SyncQueue<XrdCl::JobManager::JobHelper>::Get (this=0x3d37f08) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdCl/XrdClSyncQueue.hh:66
#4 XrdCl::JobManager::RunJobs (this=0x3d37ef0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:151
#5 0x00007fab95983f09 in RunRunnerThread (arg=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:34
#6 0x00007fabb4889d22 in start_thread () from /lib64/libc.so.6
#7 0x00007fabb490ed40 in clone3 () from /lib64/libc.so.6
Thread 5 (Thread 0x7fab93ffc640 (LWP 1169725) "unfold"):
#0 0x00007fabb48868ba in __futex_abstimed_wait_common () from /lib64/libc.so.6
#1 0x00007fabb4891d68 in __new_sem_wait_slow64.constprop.0 () from /lib64/libc.so.6
#2 0x00007fab95983e56 in XrdSysSemaphore::Wait (this=0x3bc3ec0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdSys/XrdSysPthread.hh:509
#3 XrdCl::SyncQueue<XrdCl::JobManager::JobHelper>::Get (this=0x3d37f08) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdCl/XrdClSyncQueue.hh:66
#4 XrdCl::JobManager::RunJobs (this=0x3d37ef0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:151
#5 0x00007fab95983f09 in RunRunnerThread (arg=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:34
#6 0x00007fabb4889d22 in start_thread () from /lib64/libc.so.6
#7 0x00007fabb490ed40 in clone3 () from /lib64/libc.so.6
Thread 4 (Thread 0x7fab947fd640 (LWP 1169724) "unfold"):
#0 0x00007fabb48868ba in __futex_abstimed_wait_common () from /lib64/libc.so.6
#1 0x00007fabb4891d68 in __new_sem_wait_slow64.constprop.0 () from /lib64/libc.so.6
#2 0x00007fab95983e56 in XrdSysSemaphore::Wait (this=0x3bc3ec0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdSys/XrdSysPthread.hh:509
#3 XrdCl::SyncQueue<XrdCl::JobManager::JobHelper>::Get (this=0x3d37f08) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdCl/XrdClSyncQueue.hh:66
#4 XrdCl::JobManager::RunJobs (this=0x3d37ef0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:151
#5 0x00007fab95983f09 in RunRunnerThread (arg=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClJobManager.cc:34
#6 0x00007fabb4889d22 in start_thread () from /lib64/libc.so.6
#7 0x00007fabb490ed40 in clone3 () from /lib64/libc.so.6
Thread 3 (Thread 0x7fab94ffe640 (LWP 1169723) "unfold"):
#0 0x00007fabb48d4175 in clock_nanosleep
GLIBC_2.2.5 () from /lib64/libc.so.6
#1 0x00007fabb48d8d87 in nanosleep () from /lib64/libc.so.6
#2 0x00007fabac6fa1c8 in XrdSysTimer::Wait (mills=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdSys/XrdSysTimer.cc:239
#3 0x00007fab959033c6 in XrdCl::TaskManager::RunTasks (this=0x3d5d5a0) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClTaskManager.cc:246
#4 0x00007fab959034c9 in RunRunnerThread (arg=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdCl/XrdClTaskManager.cc:38
#5 0x00007fabb4889d22 in start_thread () from /lib64/libc.so.6
#6 0x00007fabb490ed40 in clone3 () from /lib64/libc.so.6
Thread 2 (Thread 0x7fab957ff640 (LWP 1169722) "unfold"):
#0 0x00007fabb490e37e in epoll_wait () from /lib64/libc.so.6
#1 0x00007fabac6f47ba in XrdSys::IOEvents::PollE::Begin (this=0x3d7c020, syncsem=<optimized out>, retcode=<optimized out>, eTxt=<optimized out>) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/./XrdSys/XrdSysIOEventsPollE.icc:212
#2 0x00007fabac6f094d in XrdSys::IOEvents::BootStrap::Start (parg=0x7ffdfe4dc060) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdSys/XrdSysIOEvents.cc:149
#3 0x00007fabac6f98d7 in XrdSysThread_Xeq (myargs=0x1aba630) at /build/jenkins/workspace/lcg_release_pipeline/build/externals/xrootd-5.5.4/src/xrootd/5.5.4/src/XrdSys/XrdSysPthread.cc:86
#4 0x00007fabb4889d22 in start_thread () from /lib64/libc.so.6
#5 0x00007fabb490ed40 in clone3 () from /lib64/libc.so.6
Thread 1 (Thread 0x7fabb5df4c40 (LWP 1169531) "unfold"):
#0 0x00007fabb48d8b3f in wait4 () from /lib64/libc.so.6
#1 0x00007fabb484b283 in do_system () from /lib64/libc.so.6
#2 0x00007fabb5712439 in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/views/LCG_104/x86_64-el9-gcc13-opt/lib/libCore.so
#3 0x00007fabb5711df4 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/views/LCG_104/x86_64-el9-gcc13-opt/lib/libCore.so
#4 <signal handler called>
#5 0x00000000004169de in DAS::Unfolding::unfold (inputData=..., inputMC=..., output=..., config=..., steering=16, slice=...) at /eos/user/e/euslan/EL9/DasAnalysisSystem/Core/Unfolding/bin/unfold.cc:691
#6 0x00000000004114ab in main (argc=5, argv=0x7ffdfe4ddf48) at /eos/user/e/euslan/EL9/DasAnalysisSystem/Core/Unfolding/bin/unfold.cc:1011
===========================================================
The lines below might hint at the cause of the crash. If you see question
marks as part of the stack trace, try to recompile with debugging information
enabled and export CLING_DEBUG=1 environment variable before running.
You may get help by asking at the ROOT forum https://root.cern/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5 0x00000000004169de in DAS::Unfolding::unfold (inputData=..., inputMC=..., output=..., config=..., steering=16, slice=...) at /eos/user/e/euslan/EL9/DasAnalysisSystem/Core/Unfolding/bin/unfold.cc:691
#6 0x00000000004114ab in main (argc=5, argv=0x7ffdfe4ddf48) at /eos/user/e/euslan/EL9/DasAnalysisSystem/Core/Unfolding/bin/unfold.cc:1011
===========================================================
Activity
added changepatch statusongoing typesoftware labels
assigned to @paconnor
assigned to @euslan
The problem that happens here is that you have twice the same JES variations in data and MC. In principle, this was solved in in !195 (merged).
Now, it could be useful to prevent such issues from happening in
unfold. I think that in the code, before callingmkdir, one could check if a directory of the same name already exists usingTFile::GetDirectory(): if the output is a nullptr, then it means that the directory does now exist. Else one shouldcontinue. Can you prepare a MR?mentioned in merge request !206 (merged)
mentioned in merge request !202 (merged)
added statuson hold label and removed statusongoing label
Please register or sign in to reply