DD4HEP breaks sanitizer builds in listcomponents

dd4hep has broken our sanitizer builds. see e.g. the Detector tests at

https://lhcb-nightlies.web.cern.ch/nightly/lhcb-sanitizers/1012/

These builds have proved very useful in helping identify issues in the past, so we (I) will want to, eventually restore them with a dd4hep stack build.

My best guess at the problem is the fact we are mixing a non-sanitizer enabled build of dd4hep, with a sanitizer enabled build of the LHCb stack. due to the way dd4hep does it memory management, and the specific tricks the sanitizer builds does to track things, this appears to be causing issues. Historically we have always done this, mixed non-sanitizer enable builds of the LCG deps with the LHCb stack, but until now it has not caused problems beyond some issues in the memory leak tests where it limits the usefulness of the tests, because the full memory stack cannot be tracked.

One way to test/fix this would be to try using sanitizer enabled builds of dd4hep itself.

@bcouturi @clemenci FYI Do you think its possible totry the above, i.e. roll our own sanitizer enabled builds of dd4hep to try and use in the lhcb-sanitizer builds ?

Command and error details

(build)$ env LD_LIBRARY_PATH=.:/cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/catboost/0.26.1-86407/x86_64-centos7-gcc11-dbg/catboost/libs/model_interface:/cvmfs/lhcb.cern.ch/lib/lcg/releases/GitCondDB/0.2.0-1c8c6/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/crmc/1.8.0.lhcb-4e718/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/qd/2.3.13-2f0b8/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/starlight/r313-eaf9e/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/photos++/3.56.lhcb1-50520/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/pythia6/427.2.lhcb-efc51/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/tauola++/1.1.6b.lhcb-e6a73/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/gosam_contrib/2.0-779ba/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/njet/2.0.0-560c4/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/pythia8/244.lhcb4-983d5/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/feynhiggs/2.10.2-461cb/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/powheg-box-v2/r3744.lhcb2.rdynamic-efede/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/lhapdf/6.2.3-adb82/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/rivet/3.1.4-326cb/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/yoda/1.9.0-7b852/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/openloops/2.1.2-e0cce/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Grid/srm-ifce/1.13.0-0-be254/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Grid/gfal/1.13.0-0-6fc75/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Grid/dcap/2.47.7-1-cdd28/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libunwind/1.3.1-2fa0a/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/zlib/1.2.11-8af4c/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/zeromq/4.3.4-acdd8/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/yamlcpp/0.6.3-d05b2/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/xrootd/5.3.1-e6c66/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/xqilla/2.3.4-c5cd1/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/xgboost/0.90-931ef/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/XercesC/3.2.3-714f6/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/xapian/1.4.18-04e2c/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/vdt/0.4.3-992df/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/tricktrack/1.0.9-829d0/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/tensorflow/2.5.0-a17ed/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2020_U2-daa7e/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/SIO/00.01.00-6e311/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/sqlite/3320300-3c47f/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Python/3.9.6-b0f98/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/podio/00.13-bb475/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/png/1.6.37-5186a/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/pixman/0.38.4-46112/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/pcre/8.43-5c0c1/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/pango/1.40.13-dc9ba/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/oracle/19.11.0.0.0-69dfe/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/mysql/10.4.20-c0154/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/msgpackc/3.2.0-c4011/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/lz4/1.9.2-9bdfe/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCIO/02.16.01-c3a55/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libxslt/1.1.34-a0f53/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libxml2/2.9.10-9cd93/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libxkbcommon/0.7.1-8c5b7/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libtool/2.4.2-9ad34/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libsodium/1.0.18-0b20d/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libgit2/1.1.1-942f6/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libffi/3.2.1-26487/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/libaio/0.3.110-1-3a85a/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/jemalloc/5.2.1-8154a/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/hdf5/1.10.6-a3b1e/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/HepPDT/2.06.01-dba76/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/harfbuzz/1.9.0-54c61/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/GSL/2.7-30ba4/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/graphviz/2.40.1-11724/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/gperftools/2.9.1-f56d9/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/gl2ps/1.4.0-2afdb/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/gdb/10.2-4c0be/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/freetype/2.10.0-08d63/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/fontconfig/2.13.1-241fb/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/flex/2.6.4-4c6ad/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/fmt/7.1.3-18428/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/fjcontrib/1.044-21339/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/fftw3/3.3.8-102c2/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/fastjet/3.4.0-ddbb2/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/expat/2.2.6-ad2fa/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/delphes/3.5.0-f76d7/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Davix/0.7.6-9e4c2/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/CppUnit/1.14.0-b79b3/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/clhep/2.4.4.0-b903d/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/cfitsio/3.48-e4bb8/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/cairo/1.17.2-f2c05/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/Boost/1.77.0-fd33b/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/blas/0.3.17.openblas-e5f0c/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/assimp/5.0.1-2e01c/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/EDM4hep/00.03-9895b/x86_64-centos7-gcc11-dbg/lib64:/cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/HepMC/2.06.11-d5a39/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/RELAX/root6-8bf73/x86_64-centos7-gcc11-dbg/lib:/cvmfs/lhcb.cern.ch/lib/lcg/releases/gcc/11.3.0-ad0f5/x86_64-centos7/lib64 /cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/bin/listcomponents_dd4hep -o build/lib/DetectorTestPlugins.components libDetectorTestPlugins.so 

 *** Break *** segmentation violation




===========================================================
There was a crash (#5 0x00007f04d8a7738a in SigHandler(ESignals) () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so).
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f04db07246c in waitpid () from /lib64/libc.so.6
#1  0x00007f04dafeff62 in do_system () from /lib64/libc.so.6
#2  0x00007f04d8a7b18e in TUnixSystem::Exec(char const*) () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so
#3  0x00007f04d8a7ba2f in TUnixSystem::StackTrace() () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so
#4  0x00007f04d8a7f41e in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so
#5  0x00007f04d8a7738a in SigHandler(ESignals) () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so
#6  0x00007f04d8a7f321 in sighandler(int) () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.24.08-8096f/x86_64-centos7-gcc11-dbg/lib/libCore.so
#7  <signal handler called>
#8  0x00007f04d9d1a96b in dd4hep::xml::Tag_t::Tag_t(char const*) () from build/lib/libDetectorLib.so
#9  0x00007f04d9ac21e0 in __static_initialization_and_destruction_0 () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib/libDDCond.so.1.23
#10 0x00007f04d9ac26c9 in _GLOBAL__sub_I_ConditionsTags.cpp () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib/libDDCond.so.1.23
#11 0x00007f04dbcb89b3 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#12 0x00007f04dbcbd58e in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#13 0x00007f04dbcb87c4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#14 0x00007f04dbcbcb7b in _dl_open () from /lib64/ld-linux-x86-64.so.2
#15 0x00007f04dbaa5fab in dlopen_doit () from /lib64/libdl.so.2
#16 0x00007f04dbcb87c4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#17 0x00007f04dbaa65ad in _dlerror_run () from /lib64/libdl.so.2
#18 0x00007f04dbaa6041 in dlopen
GLIBC_2.2.5 () from /lib64/libdl.so.2
#19 0x0000000000409f2e in main ()
===========================================================
The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#8  0x00007f04d9d1a96b in dd4hep::xml::Tag_t::Tag_t(char const*) () from build/lib/libDetectorLib.so
#9  0x00007f04d9ac21e0 in __static_initialization_and_destruction_0 () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib/libDDCond.so.1.23
#10 0x00007f04d9ac26c9 in _GLOBAL__sub_I_ConditionsTags.cpp () from /cvmfs/lhcb.cern.ch/lib/lcg/releases/DD4hep/01.23-a49d3/x86_64-centos7-gcc11-dbg/lib/libDDCond.so.1.23
#11 0x00007f04dbcb89b3 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#12 0x00007f04dbcbd58e in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#13 0x00007f04dbcb87c4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#14 0x00007f04dbcbcb7b in _dl_open () from /lib64/ld-linux-x86-64.so.2
#15 0x00007f04dbaa5fab in dlopen_doit () from /lib64/libdl.so.2
#16 0x00007f04dbcb87c4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#17 0x00007f04dbaa65ad in _dlerror_run () from /lib64/libdl.so.2
#18 0x00007f04dbaa6041 in dlopen
GLIBC_2.2.5 () from /lib64/libdl.so.2
#19 0x0000000000409f2e in main ()
===========================================================
[command exited with 139]