Skip to content

Multithreading: Separate Thread Initialization from Run()

Simon Spannagel requested to merge mt_thread_init into multithreading

I have experienced issues in running few events with DepositionGeant4 in a multithreaded environment. Among other issues I ran into stack traces like the following (abbreviated):

-------- EEEE ------- G4Exception-START -------- EEEE -------
*** G4Exception : InvalidCondition
      issued by : ParticlesWorspacePool::CreateWorkspace
Cannot create workspace twice for the same thread.
*** Fatal Exception *** core dump ***
G4WT42 >  **** Step information is not available at this moment
G4WT41 >  **** Step information is not available at this moment
G4WT40 >
-------- EEEE -------- G4Exception-END --------- EEEE -------

G4WT42 > *** G4Exception: Aborting execution ***

Thread 13 "allpix" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffc0aa6700 (LWP 13389)]
0x00007fffe088f387 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007fffe088f387 in raise () from /lib64/libc.so.6
#1  0x00007fffe0890a78 in abort () from /lib64/libc.so.6
#2  0x00007fffe7a60093 in G4Exception(char const*, char const*, G4ExceptionSeverity, char const*) () from /cvmfs/sft.cern.ch/lcg/views/LCG_97python3/x86_64-centos7-gcc8-opt/lib64/libG4global.so
#3  0x00007fffeefce564 in G4WorkerThread::BuildGeometryAndPhysicsVector() () from /cvmfs/sft.cern.ch/lcg/views/LCG_97python3/x86_64-centos7-gcc8-opt/lib64/libG4run.so
#4  0x00007fffe61c6008 in allpix::WorkerRunManager::GetNewInstanceForThread () at /data/simonspa/allpix-squared/src/tools/geant4/WorkerRunManager.cpp:188
#5  0x00007fffe61c4a58 in allpix::MTRunManager::InitializeForThread (this=this@entry=0x25847b0) at /data/simonspa/allpix-squared/src/tools/geant4/MTRunManager.cpp:86
#6  0x00007ffff7313040 in allpix::DepositionGeant4Module::run (this=0x255e800, event=0x7fff5c0008d0) at /data/simonspa/allpix-squared/src/modules/DepositionGeant4/DepositionGeant4Module.cpp:292

I traced this back to tow (separate) issues:

Firstly, some unclean initialization of Geant4 worker threads, mostly appearing when number_of_events < workers. Therefore I decided to introduce another stage in the succession of module commands: initializeThread(), the equivalent to finalizeThread() processed before the run() function. It can be used to properly initialize thread-local things (like our WorkerRunManagers) before events are generated and processed.

Secondly, I realized that the ThreadPool did not catch exceptions from the finalize and initialize functions provided to it. I changed this, so the full lifetime of the worker thread is now covered.

Merge request reports