Gauss port to aarch64 (using LCG102b)
I will document here a few points from my attempt to help in the port of Gauss to aarch64. Copying @clemenci and @bcouturi.
This is based on Marco's https://codimd.web.cern.ch/s/G2qnvDjev using LCG102b on centos7.
Designs
- lhcb/upgrade-hackathon-setup !54
Activity
As per Marco's instructions, this is based on an el7 container from an el8 or el9 host node.
- Initially I started this on lbbuildinter02. This was ok until I had to pull external libraries (OpenGL for Geant4, via HEP_OSlibs) inside my container. Then I realised that lbbuildinter02 (the host, not the container) has network issues, it cannot ping mirrorlist.centos.org.
- Therefore I moved to an openlab node, olarm-203. The rest of what I say below is on that node (unless otherwise stated).
The points I describe below are some issues and how I worked around them to advance. A real fix is in most cases still needed.
First, the part before Gauss.
- For Gaudi and GitCondDB, I disabled Intel amplifier and I added GitCondDB to the list of projects
After
cp -dpr configuration.mk configuration.mk.ORIGINAL cat configuration.mk.ORIGINAL | sed "s/ -DGAUDI_USE_INTELAMPLIFIER=YES//" | sed "s/PROJECTS = Gaudi Detector/PROJECTS = Gaudi GitCondDB Detector/" > configuration.mkI was then able to
make Gaudi make GitCondDBNote, on the previous machine lbbuildinter02 I had tried
make Gaudi/testand this gave three failed tests.- For Detector and LHCb I added Marco's patches
These are
- for Detector: Detector!334 (merged)
- for LHCb: LHCb!3910 (merged)
I did
make Detectorwhich failed. This fixed itcd Detector git cherry-pick 3f06380f cd .. make DetectorI did
make LHCbwhich failed. This fixed itcd LHCb git cherry-pick 80474216 9659f188 4bc6e7ba cd .. make LHCbThen this went through out of the box
make Run2SupportEdited by Andrea ValassiI'd like to point out that my instructions in https://codimd.web.cern.ch/s/G2qnvDjev describe the changes needed in
configuration.mkas:echo platform=armv8.1_a-centos7-gcc11-opt >> workspace/configuration.mk echo LCG_VERSION=102b >> workspace/configuration.mk echo WITH_GITCONDDB = 1 >> workspace/configuration.mkThe test failures in Gaudi require some update of the stdout reference files, which I made in gaudi/Gaudi!1396 (merged) before deciding to use
armv8.1_ainstead ofarmv8_a.About IntelAplifier, it should not be mentioned in
configuration.mk. I did the same as you in my latest tests, but I forgot to update the CodiMD page.In any case I just created upgrade-hackathon-setup!54 (merged).
Ciao Marco, thanks a lot.
I am now able to follow your codimd instructions to the end on lbbuildinter02, using podman in /home/andreav an dusing your latest container with the new HEP_OSlibs.
-
About 102b, my bad, sorry: I was still using 101arm_3 which was probably a previous version of codimd...
-
About the Gaudi tests: thanks, ok so the failures are expected becuse the ref files must be updated. I confirm that I get these
Total Test time (real) = 878.83 sec The following tests did not run: 206 - GaudiExamples.google_auditors.heapchecker (Skipped) 225 - GaudiExamples.jira.gaudi_1174 (Disabled) 278 - GaudiExamples.skipped_test (Skipped) The following tests FAILED: 209 - GaudiExamples.histoex (Failed) 211 - GaudiExamples.histoex2 (Failed) 265 - GaudiExamples.root_io.extcoll.read (Failed)-
About IntelAmplifier: very good, thanks. With your new hackathon setup I do not need to change it in configuration.mk.
-
About GitCondDB: I confirm that I am using your modification line in configuration.mk, but this is not enough. I cannot
make Detectorormake GitCondDBunless/until I do the following
echo PROJECTS += GitCondDB >> configuration.mkThanks to you and @sponce for merging the patches to Detector and LHCb. I am now able to build GitCondDB, Detector, LHCb and Run2Support. I also tried the tests, but I am not sure they should work (do I need to specify some geometry tag to use? it seems to create one). The GitCondDB tests succceed, but the Detector, LHCb and Run2Support tests fail. For
make Detector/test:cd /workspace/Detector/build.armv8.1_a-centos7-gcc11-opt && ctest -T test Site: lbbuildinter02.cern.ch Build name: Linux-/workspace/Detector/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++-Release Create new tag: 20230110-1306 - Experimental Test project /workspace/Detector/build.armv8.1_a-centos7-gcc11-opt ... Total Test time (real) = 59.11 sec The following tests FAILED: 10 - Detector.Core.DDS with overlay (Failed) 12 - test_load_all (Failed) 13 - test_load_VP_debug (Failed) 14 - test_load_UT_debug (Failed) 15 - test_load_FT_debug (Failed) 16 - test_load_Rich1_debug (Failed) 17 - test_load_Rich2_debug (Failed) 18 - test_load_Muon_debug (Failed) 19 - test_load_Ecal_debug (Failed) 20 - test_load_Hcal_debug (Failed) 21 - test_find_intersection (Failed) 22 - test_load_conditions (Failed) 23 - test_vp_ladder_position (Failed) 24 - test_vp_motion (Failed) 25 - test_vp_sensor_position (Failed) 26 - test_Magnet_load_condition (Failed) 27 - test_Magnet_load_field (Failed) 28 - test_Magnet_load_demagnet (Failed) 29 - test_load_deft (Failed) 30 - test_load_deft_cond (Failed) 31 - test_load_deut (Failed) 32 - test_load_devp (Failed) 33 - test_load_derich1 (Failed) 34 - test_load_derich2 (Failed) 35 - test_load_demuon (Failed) 36 - test_load_delhcb (Failed) 37 - test_load_devp_detailed (Failed) 38 - test_load_demuon_detailed (Failed) 39 - AlignmentUseCase (Failed) 45 - test_dump_Rich1 (Failed) 46 - test_dump_Rich2 (Failed) 52 - Detector.Muon.DeMuon (Failed) Some tests failed: FAIL: 32 ERROR: 0And for
make LHCb/test:Total Test time (real) = 1294.81 sec The following tests did not run: 7 - DAQSys.gaudirun-defsplit (Disabled) 8 - DAQSys.gaudirun (Disabled) 29 - UTDet.test_deut (Skipped) 44 - DetCond.condition_accessor.dd4hep_check_db_access (Disabled) 45 - DetCond.condition_accessor.dd4hep_check_db_access_bare (Disabled) 46 - DetCond.condition_accessor.dd4hep_cond_as_json (Disabled) 47 - DetCond.condition_accessor.dd4hep_cond_as_yaml (Disabled) 48 - DetCond.condition_accessor.dd4hep_cond_no_prop (Disabled) 49 - DetCond.condition_accessor.dd4hep_conditions (Disabled) 50 - DetCond.condition_accessor.dd4hep_conditions_error (Disabled) 51 - DetCond.condition_accessor.dd4hep_override_conditions (Disabled) 52 - DetCond.condition_accessor.dd4hep_sharedconditions (Disabled) 140 - HltDAQ.b_dumpafterreformat (Disabled) 252 - RichDetectors.test-run-change-2022-data (Disabled) The following tests FAILED: 1 - CaloFutureDAQ.calo_decoding (Failed) 2 - DAQSys.calosegfault (Failed) 6 - DAQSys.gaudirun-decoderaweventpy (Failed) 9 - DAQSys.mdfdump (Failed) 10 - DAQSys.mdfwriter (Failed) 11 - DAQSys.split-raw-event (Failed) 13 - DAQUtils.decision_bank_killer (Failed) 14 - DAQUtils.dumpmultievents (Failed) 15 - MDF.dst_from_mdf (Failed) 16 - MDF.existing_mdf (Failed) 17 - MDF.read_dst_from_mdf (Not Run) 20 - RawEventCompat.gaudiread (Not Run) 21 - RawEventCompat.gaudirun (Failed) 26 - RawEventCompat.test-badbanks (Failed) 28 - Magnet.test_magfieldsvc (Failed) 36 - VPDet.veloaligncond.updates_yaml_pyconf (Failed) 92 - IOExample.copyinput (Failed) 93 - IOExample.copymc10dsttoroot (Failed) 94 - IOExample.copypooldigitoroot (Failed) 95 - IOExample.copypoolsimtoroot (Failed) 96 - IOExample.copyreco08dsttoroot (Failed) 97 - IOExample.copyreco12stripping17dsttoroot (Failed) 98 - IOExample.copyreco12stripping17mdsttoroot (Failed) 99 - IOExample.copyreco14stripping20dsttoroot (Failed) 100 - IOExample.copyreco14stripping20mdsttoroot (Failed) 101 - IOExample.copyreco14stripping21mdsttoroot (Failed) 102 - IOExample.mdfreadingexample (Failed) 103 - IOExample.readmc10rootdst (Not Run) 104 - IOExample.readreco08rootdst (Not Run) 105 - IOExample.readreco12stripping17dst (Not Run) 106 - IOExample.readreco12stripping17mdst (Not Run) 107 - IOExample.readreco14stripping20dst (Not Run) 108 - IOExample.readreco14stripping20mdst (Not Run) 109 - IOExample.readreco14stripping21mdst (Not Run) 110 - IOExample.readrootdigi (Not Run) 111 - IOExample.readrootsim (Not Run) 112 - IOExample.testunpacktrack (Failed) 113 - FTDAQ.scifi_decoding (Failed) 119 - GaudiConf.python (Failed) 132 - HLTScheduler.exec_reports_writer (Failed) 139 - HltDAQ.a_dumptolatestformat (Failed) 141 - HltDAQ.lumi_decoding (Failed) 142 - HltDAQ.sel_reports_stripper (Failed) 145 - HltServices.close_cdb_file (Failed) 149 - HltServices.taskid (Failed) 150 - FSRAlgs.mergesmallfiles (Failed) 152 - LHCbAlgs.fsr-only-file-root (Failed) 153 - LHCbAlgs.fsr-small-files-root (Not Run) 155 - LHCbAlgs.lbappinit (Failed) 159 - LHCbAlgs.odinemulator (Failed) 165 - LHCbAlgs.timingtabletofile (Failed) 166 - LHCbAlgs.xmlsummary (Failed) 180 - LHCbKernel.test_PlatformInfo (Failed) 202 - LHCbMath.testsimilarityaccuracy (Failed) 205 - LHCbMath.testvdtmath_AVX (Failed) 206 - LHCbMath.testvdtmath_AVX2 (Failed) 207 - LHCbMath.testvdtmath_AVX2FMA (Failed) 208 - LHCbMath.testvdtmath_SSE4 (Failed) 211 - LHCbMath.test_matvec (Failed) 223 - XMLSummaryKernel.exception (Failed) 224 - XMLSummaryKernel.exit (Failed) 226 - XMLSummaryKernel.writedst (Failed) 227 - XMLSummaryKernel.writeerror (Failed) 228 - MuonDAQ.decoding (Failed) 237 - PyConf.test_diffopts_git (Failed) 238 - PyConf.test_diffopts_ndiff (Failed) 239 - PyConf.test_diffopts_opts (Failed) 240 - PyConf.test_diffopts_pkl (Not Run) 241 - PyConf.test_example (Failed) 242 - PyConf.test_example_auditors (Failed) 243 - PyConf.test_example_pkl (Failed) 244 - PyConf.test_incomplete (Failed) 245 - PyConf.test_unpacking_busy_event (Failed) 246 - PyConf.python (Failed) 247 - RichDetectors.test-decode-and-spacepoints-2022-data (Failed) 248 - RichDetectors.test-decode-and-spacepoints-detdesc-mc (Failed) 249 - RichDetectors.test-instanciate-compat-old-rich1 (Failed) 250 - RichDetectors.test-instanciate-compat-trunk (Failed) 251 - RichDetectors.test-radiator-volume-intersections (Failed) 253 - FileStager.filestager (Failed) 256 - VPDAQ.retina_decoding (Failed) 257 - VPDAQ.retina_full_decoding (Failed) ... Some tests failed: FAIL: 69 ERROR: 0And for
make Run2Support/test:Create new tag: 20230110-1342 - Experimental Test project /workspace/Run2Support/build.armv8.1_a-centos7-gcc11-opt ... Total Test time (real) = 31.81 sec The following tests FAILED: 2 - VeloDet.veloaligncond.basic (Failed) 3 - VeloDet.veloaligncond.override1 (Failed) 4 - VeloDet.veloaligncond.override2 (Failed) 5 - VeloDet.veloaligncond.override3 (Failed) 6 - VeloDet.veloaligncond.updates (Failed) Errors while running CTest ... Some tests failed: FAIL: 5 ERROR: 0-
Next, the configuration to
make Gauss. I have still not completed this.(By the way, copying @giordano too for info).
- The "LHCB_7" layer for aarch64 is missing. (I am not sure what this is and why it is only used in Gauss).
I fixed this with
\cp /workspace/lcg-toolchains/fragments/packages/LCG_102b-aarch64-centos7-gcc11-opt.cmake \ /workspace/lcg-toolchains/fragments/packages/LCG_102b_LHCB_7-aarch64-centos7-gcc11-opt.cmake- Geant4 is not found
Fixed it by
export CMAKE_PREFIX_PATH_original=$CMAKE_PREFIX_PATH export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH_original:/cvmfs/sft.cern.ch/lcg/releases/Geant4/11.0.3-21a84/aarch64-centos7-gcc11-opt make Gauss- Geant4 cannot find OpenGL.
My understanding is that for x86 this would normally be pulled by HEP_OSlibs. I therefore installed HEP_OSlibs within the container. (This is the reason why I moved from lbbuildinter02 to olarm-203).
Note: HEP_OSlibs for ARM is going to be regularly supported for el8/el9, but for el7 it is still shaky. The version that was available yesterday for el7 did not build in my container, see linuxsupport/rpms/HEP_OSlibs#8 (closed). I have now created a new version for el7, and I was able to install it
wget https://gitlab.cern.ch/linuxsupport/rpms/HEP_OSlibs/-/raw/el7/downloads/aarch64/HEP_OSlibs-7.3.5-3.el7.cern.aarch64.rpm yum localinstall HEP_OSlibs-7.3.5-3.el7.cern.aarch64.rpmNow OpenGL is found, and indeed it comes through a library pulled in (via mesa?) by HEP_OSlibs
-- Found OpenGL: /usr/lib64/libOpenGL.so ... [root@olarm-203 bash] /workspace> rpm -qf /usr/lib64/libOpenGL.so libglvnd-devel-1.0.1-0.8.git5baa1e5.el7.aarch64Ah, one important point: if you agree that OpenGL must come from HEP_OSlibs, it would be good to add HEP_OSlibs to the container, that is Marco's gitlab-registry.cern.ch/lhcb-core/lbdocker/aarch64-centos7-build
The rest, below, is even more work in progress.
Edited by Andrea ValassiHEP_OSlibsis already included, but only the version I could find at the time, see https://gitlab.cern.ch/lhcb-core/LbDocker/-/blob/master/aarch64/Containerfile#L5For the
LHCB_7layer I didn't do anything because we do not have the ARM builds of the generators we need. Your workaround allows you to move a bit further, but we need something proper.I updated the configuration of the container (lhcb-core/LbDocker!41 (merged) + lhcb-core/LbDocker!42 (merged)) and I'll create a new version ASAP.
Edited by Marco ClemencicNew Docker/Podman image created and published. Thanks for the update of HEP_OSlibs.
Edited by Marco ClemencicThanks a lot Marco.
- About the container, I confirm that the new container has HEP_OSlibs and thus includes OpenGL. Just a minor note, it seems that podman did not look for the latest version in my first try this year, and used the cache. So maybe better update the codimd to specify "latest" or a specific version? I used this
podman run -ti --network host -v /cvmfs:/cvmfs:shared -v ${PWD}/workspace:/workspace:z -w /workspace gitlab-registry.cern.ch/lhcb-core/lbdocker/aarch64-centos7-build:2023-01-05- About LHCb_7, we should have a chat at some point how it can/should be improved. For the moment I just did the same that I had done, and actually it worked (see further comments below about CRMC), so maybe this is enough?
\cp /workspace/lcg-toolchains/fragments/packages/LCG_102b-aarch64-centos7-gcc11-opt.cmake \ /workspace/lcg-toolchains/fragments/packages/LCG_102b_LHCB_7-aarch64-centos7-gcc11-opt.cmake- Just one more point with respect to what I said above, somehow it is no longer needed to add Geant4 to the CMAKE_PREFIX_PATH (maybe thanks to me using 102b and not 101arm3...)
After the points above, the rest is even more WIP. I started adding some patches to git for simplicity.
This is a WIP MR: !914 (closed)
- CRMC is not found.
I do not know if this changed in LCG102b and/or specifically for ARM, but FindCRMC.cmake was looking for its headers in src, which is bizarre. I changed this to look for them in include/crmc instead
- Madgraph LHAPDF62/C++ is not found
I have no idea how much or where Madgraph PDFs are used now in LHCb. It seems that the C++ version was looked for, but in LCG102b/ARM only a fortran version is available. I moved to that - again, a dirty workaround, a real fix is needed.
- Photos++ CxxInterface and Fortran are missing
Here again I have no idea how much or whiche versions of Photos are used normally by LHCb. Here cmake was looking for libPhotosCxxInterface and libPhotosFortran, but LCG102b/ARM has instead libPhotospp and things like libPhotosHEPEVT. Initially I tried to change the FindPhotos.cmake to use those, but then I saw these are (not surprisingly) used elsehwre in the code. So my current workaround is to disable Photos altogether in GaussDependencies and in Gen/LbEvt.
Probably the real fix, as for Madgraph, is to get the needed files in the LCG102b/ARM on cvmfs?
- After the above workarounds, my
make Gaussis still stuck here, in LaMarr and GaussPhysics. I will probably only look at this after the winter break.
CMake Error at /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/lib/cmake/Gaudi/GaudiToolbox.cmake:399 (add_library): Target "GaussPhysics" links to target "Geant4::G4LHCblists" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing? Call Stack (most recent call first): Sim/GaussPhysics/CMakeLists.txt:16 (gaudi_add_module) CMake Error at /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/lib/cmake/Gaudi/GaudiToolbox.cmake:399 (add_library): Target "LbLamarr" links to target "LHCb::CaloUtils" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing? Call Stack (most recent call first): Sim/LbLamarr/CMakeLists.txt:16 (gaudi_add_module)Voila, that's all for now, hope it helps.
Thanks Marco. I am not sure if my Lamarr issue above is due to "an incorrect LHCB_7 definition". Actually the "Sim/GaussPhysics" issue no longer appears. As for Lamarr, I fixed it by simply disabling LHCb::CaloUtils in Lamarr, see !914 (54537d09). It seems that
git clone LHCbpulls in Calo/CaloKernel but not Calo/CaloUtils, so I assume this is simply not needed anymore?
General status update, after several in-line replies. I have restarted from scratch, with several improvements thanks to Marco.
I now am using lbbuildinter02 because Marco's new container includes HEP_OSlibs already (that I need for OpenGL), so I do not need to pull it (which would fail on lbbuildinter02 as I have no network apparently).
I had a mistake in December in my configuration.mk, now fixed thanks to Marco (use LCG102b there).
I still have to do a few hacks as described above, mainly
- Add
echo PROJECTS += GitCondDB >> configuration.mk - Create LHCB_7, by copying as-is the LCG_102b-aarch64-centos7-gcc11-opt.cmake toolchain to LCG_102b_LHCB_7-aarch64-centos7-gcc11-opt.cmake
- In addition, I still need all the configuration patches to Gauss in MR !914 (closed) . This includes so far the seven commits in December plus two additional ones I just added, one for Lamarr (disable LHCb/Calo/CaloUtils) and one for CRMC (to find th elib directory in cmake)
Once the above things are done, the Gauss configuration is fixed "enough" to start the build. A lot of stuff builds, but then there are a few build errors in the source code that I need to fix.
If I do
make Gaussover and over (so that the parallel build manages to do as much as possible even if elsewhere there are errors), the main issues to fix seem to be the following- Build flags -march in AmpGen (this is a real aarch64 issue)
[1/82] Building CXX object CMakeFiles/AmpGen.exe.dir/apps/AmpGen.cpp.o FAILED: CMakeFiles/AmpGen.exe.dir/apps/AmpGen.cpp.o /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++ -DENABLE_AVX=0 -DENABLE_FITTING=0 -DUSE_MVEC=0 -I/cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-ec51b/aarch64-centos7-gcc11-opt/include -I/workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/Gen/LbAmpGen/AmpGenGitHub-prefix/src/AmpGenGitHub -I. -I/cvmfs/sft.cern.ch/lcg/releases/GSL/2.7-30ba4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-adcbe/aarch64-centos7-gcc11-opt/include -O3 -DNDEBUG -g3 -O3 -Wall -Wextra -Wpedantic -Wno-unused-parameter -Wno-unknown-pragmas -march=x86-64 -Wno-suggest-override -fopenmp -std=c++17 -MD -MT CMakeFiles/AmpGen.exe.dir/apps/AmpGen.cpp.o -MF CMakeFiles/AmpGen.exe.dir/apps/AmpGen.cpp.o.d -o CMakeFiles/AmpGen.exe.dir/apps/AmpGen.cpp.o -c /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/Gen/LbAmpGen/AmpGenGitHub-prefix/src/AmpGenGitHub/apps/AmpGen.cpp cc1plus: error: unknown value 'x86-64' for '-march' cc1plus: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8-r nativeThere are around 82 occurrences of this error.
- The Photos header is not found
[11/68] Building CXX object Gen/LbEvtGen/CMakeFiles/LbEvtGenLib.dir/src/Lib/ApplyPhotos.cpp.o FAILED: Gen/LbEvtGen/CMakeFiles/LbEvtGenLib.dir/src/Lib/ApplyPhotos.cpp.o ccache /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SPIRIT_USE_PHOENIX_V3 -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DDD4HEP_USE_TBB -DDD4HEP_USE_XERCESC -DFMT_LOCALE -DFMT_SHARED -DGAUDI_V20_COMPAT -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DLbEvtGenLib_EXPORTS -I/cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/lib/CLHEP-2.4.5.1/../../include -I../Gen/LbEvtGen/include -I../Gen/LbEvtGen/src/include -I../Gen/Mint/include -I../Gen/Generators/include -I../Gen/GENSER/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Boost/1.78.0-479ff/aarch64-centos7-gcc11-opt/include -isystem Gen/LbEvtGen/EvtGenHepForge-prefix/src/EvtGenHepForge -isystem /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-ec51b/aarch64-centos7-gcc11-opt/include -isystem /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-adcbe/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/cppgsl/3.1.0-7a1b6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/fmt/7.1.3-18428/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/jsonmcpp/3.10.5-f26c3/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/AIDA/3.2.1-3fe9f/aarch64-centos7-gcc11-opt/src/cpp -isystem /cvmfs/sft.cern.ch/lcg/releases/rangev3/0.11.0-79ab4/aarch64-centos7-gcc11-opt/include -isystem /workspace/LHCb/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/HepMC/2.06.11-d5a39/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/eigen/3.3.7-642a5/aarch64-centos7-gcc11-opt/include/eigen3 -isystem /cvmfs/sft.cern.ch/lcg/releases/GSL/2.7-30ba4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Vc/1.4.3-57098/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/vdt/0.4.3-7622c/aarch64-centos7-gcc11-opt/include -isystem /workspace/yaml-cpp/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/Detector/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/DD4hep/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/XercesC/3.2.3-714f6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia6/429.2-5d3d1/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/lhapdf/6.5.3-d2254/aarch64-centos7-gcc11-opt/include -march=armv8.1-a -fmessage-length=0 -pipe -fdiagnostics-color -pedantic -Wall -Wextra -Werror=return-type -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wnon-virtual-dtor -Wsuggest-override -O3 -DNDEBUG -fPIC -pthread -std=gnu++17 -MD -MT Gen/LbEvtGen/CMakeFiles/LbEvtGenLib.dir/src/Lib/ApplyPhotos.cpp.o -MF Gen/LbEvtGen/CMakeFiles/LbEvtGenLib.dir/src/Lib/ApplyPhotos.cpp.o.d -o Gen/LbEvtGen/CMakeFiles/LbEvtGenLib.dir/src/Lib/ApplyPhotos.cpp.o -c ../Gen/LbEvtGen/src/Lib/ApplyPhotos.cpp ../Gen/LbEvtGen/src/Lib/ApplyPhotos.cpp:13:10: fatal error: Photos/Photos.h: No such file or directory 13 | #include "Photos/Photos.h" | ^~~~~~~~~~~~~~~~~ compilation terminated.- One CaloUtils header is not found. So my patch above must be undone. But where is CaloUtils supposed to come from, is it LHCb/Calo/CaloUtils? If this is the case, why is it not downloaded by
git clone LHCb?
[12/68] Building CXX object Sim/LbLamarr/CMakeFiles/LbLamarr.dir/src/LamarrCaloProto.cpp.o FAILED: Sim/LbLamarr/CMakeFiles/LbLamarr.dir/src/LamarrCaloProto.cpp.o ccache /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTAINER_DYN_LINK -DBOOST_CONTAINER_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SPIRIT_USE_PHOENIX_V3 -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DDD4HEP_USE_TBB -DDD4HEP_USE_XERCESC -DFMT_LOCALE -DFMT_SHARED -DGAUDI_V20_COMPAT -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DLbLamarr_EXPORTS -I/cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/lib/CLHEP-2.4.5.1/../../include -isystem /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-ec51b/aarch64-centos7-gcc11-opt/include -isystem /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Boost/1.78.0-479ff/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-adcbe/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/cppgsl/3.1.0-7a1b6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/fmt/7.1.3-18428/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/jsonmcpp/3.10.5-f26c3/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/AIDA/3.2.1-3fe9f/aarch64-centos7-gcc11-opt/src/cpp -isystem /cvmfs/sft.cern.ch/lcg/releases/rangev3/0.11.0-79ab4/aarch64-centos7-gcc11-opt/include -isystem /workspace/LHCb/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/eigen/3.3.7-642a5/aarch64-centos7-gcc11-opt/include/eigen3 -isystem /cvmfs/sft.cern.ch/lcg/releases/GSL/2.7-30ba4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Vc/1.4.3-57098/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/vdt/0.4.3-7622c/aarch64-centos7-gcc11-opt/include -isystem /workspace/Detector/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/DD4hep/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/XercesC/3.2.3-714f6/aarch64-centos7-gcc11-opt/include -isystem /workspace/yaml-cpp/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/HepMC/2.06.11-d5a39/aarch64-centos7-gcc11-opt/include -march=armv8.1-a -fmessage-length=0 -pipe -fdiagnostics-color -pedantic -Wall -Wextra -Werror=return-type -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wnon-virtual-dtor -Wsuggest-override -O3 -DNDEBUG -fPIC -pthread -std=gnu++17 -MD -MT Sim/LbLamarr/CMakeFiles/LbLamarr.dir/src/LamarrCaloProto.cpp.o -MF Sim/LbLamarr/CMakeFiles/LbLamarr.dir/src/LamarrCaloProto.cpp.o.d -o Sim/LbLamarr/CMakeFiles/LbLamarr.dir/src/LamarrCaloProto.cpp.o -c ../Sim/LbLamarr/src/LamarrCaloProto.cpp In file included from ../Sim/LbLamarr/src/LamarrCaloProto.cpp:32: ../Sim/LbLamarr/src/LamarrCaloProto.h:25:10: fatal error: CaloUtils/CaloAlgUtils.h: No such file or directory 25 | #include "CaloUtils/CaloAlgUtils.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.- A build error related to Pythia and SuperChic
[13/68] Building CXX object Gen/LbSuperChic2/CM...erChic2.dir/src/component/LHAupSuperChic2.cpp.o FAILED: Gen/LbSuperChic2/CMakeFiles/LbSuperChic2.dir/src/component/LHAupSuperChic2.cpp.o ccache /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SPIRIT_USE_PHOENIX_V3 -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DDD4HEP_USE_TBB -DDD4HEP_USE_XERCESC -DFMT_LOCALE -DFMT_SHARED -DGAUDI_V20_COMPAT -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DLbSuperChic2_EXPORTS -I/cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/lib/CLHEP-2.4.5.1/../../include -I../Gen/SuperChic2/include -I../Gen/Generators/include -I../Gen/GENSER/include -I../Gen/LbHard/include -I../Gen/LbPythia/include -I../Gen/LbPythia8/include -isystem /workspace/LHCb/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Boost/1.78.0-479ff/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-adcbe/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-ec51b/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/cppgsl/3.1.0-7a1b6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/fmt/7.1.3-18428/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/jsonmcpp/3.10.5-f26c3/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/HepMC/2.06.11-d5a39/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/AIDA/3.2.1-3fe9f/aarch64-centos7-gcc11-opt/src/cpp -isystem /cvmfs/sft.cern.ch/lcg/releases/rangev3/0.11.0-79ab4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/eigen/3.3.7-642a5/aarch64-centos7-gcc11-opt/include/eigen3 -isystem /cvmfs/sft.cern.ch/lcg/releases/GSL/2.7-30ba4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Vc/1.4.3-57098/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/vdt/0.4.3-7622c/aarch64-centos7-gcc11-opt/include -isystem /workspace/yaml-cpp/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/Detector/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/DD4hep/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/XercesC/3.2.3-714f6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia6/429.2-5d3d1/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/lhapdf/6.5.3-d2254/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include -march=armv8.1-a -fmessage-length=0 -pipe -fdiagnostics-color -pedantic -Wall -Wextra -Werror=return-type -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wnon-virtual-dtor -Wsuggest-override -O3 -DNDEBUG -fPIC -pthread -std=gnu++17 -MD -MT Gen/LbSuperChic2/CMakeFiles/LbSuperChic2.dir/src/component/LHAupSuperChic2.cpp.o -MF Gen/LbSuperChic2/CMakeFiles/LbSuperChic2.dir/src/component/LHAupSuperChic2.cpp.o.d -o Gen/LbSuperChic2/CMakeFiles/LbSuperChic2.dir/src/component/LHAupSuperChic2.cpp.o -c ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp In file included from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Analysis.h:16, from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Pythia.h:17, from ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.h:19, from ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp:15: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Basics.h: In member function 'std::pair<double, double> Pythia8::Rndm::gauss2()': /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Basics.h:411:33: note: parameter passing for argument of type 'std::pair<double, double>' when C++17 is enabled changed to match C++14 in GCC 10.1 411 | pair<double, double> gauss2() {double r = sqrt(-2. * log(flat())); | ^ ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp: In member function 'virtual bool Pythia8::LHAupSuperChic2::setInit()': ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp:208:11: error: 'class Pythia8::Pythia' has no member named 'couplings' 208 | pythia->couplings.init(pythia->settings, &pythia->rndm); | ^~~~~~~~~ ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp:209:33: error: 'class Pythia8::Pythia' has no member named 'couplings' 209 | ewpars_.gf[0] = pythia->couplings.GF(); | ^~~~~~~~~ ../Gen/LbSuperChic2/src/component/LHAupSuperChic2.cpp:226:33: error: 'class Pythia8::Pythia' has no member named 'couplings' 226 | ewpars_.alpha[0] = pythia->couplings.alphaEM(0); | ^~~~~~~~~- Another error related to Pythia
[14/68] Building CXX object Gen/LbPythia8/CMake...bPythia8Lib.dir/src/Lib/Pythia8Production.cpp.o FAILED: Gen/LbPythia8/CMakeFiles/LbPythia8Lib.dir/src/Lib/Pythia8Production.cpp.o ccache /workspace/Gauss/build.armv8.1_a-centos7-gcc11-opt/toolchain/g++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SPIRIT_USE_PHOENIX_V3 -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DDD4HEP_USE_TBB -DDD4HEP_USE_XERCESC -DFMT_LOCALE -DFMT_SHARED -DGAUDI_V20_COMPAT -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DLbPythia8Lib_EXPORTS -I/cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/lib/CLHEP-2.4.5.1/../../include -I../Gen/LbPythia8/include -I../Gen/Generators/include -I../Gen/GENSER/include -isystem /workspace/LHCb/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/Gaudi/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Boost/1.78.0-479ff/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/tbb/2020_U2-adcbe/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-ec51b/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/cppgsl/3.1.0-7a1b6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/fmt/7.1.3-18428/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/jsonmcpp/3.10.5-f26c3/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/HepMC/2.06.11-d5a39/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/AIDA/3.2.1-3fe9f/aarch64-centos7-gcc11-opt/src/cpp -isystem /cvmfs/sft.cern.ch/lcg/releases/rangev3/0.11.0-79ab4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/eigen/3.3.7-642a5/aarch64-centos7-gcc11-opt/include/eigen3 -isystem /cvmfs/sft.cern.ch/lcg/releases/GSL/2.7-30ba4/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/clhep/2.4.5.1-ebe73/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/Vc/1.4.3-57098/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/vdt/0.4.3-7622c/aarch64-centos7-gcc11-opt/include -isystem /workspace/yaml-cpp/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/Detector/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /workspace/DD4hep/InstallArea/armv8.1_a-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/XercesC/3.2.3-714f6/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia6/429.2-5d3d1/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/lhapdf/6.5.3-d2254/aarch64-centos7-gcc11-opt/include -isystem /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include -march=armv8.1-a -fmessage-length=0 -pipe -fdiagnostics-color -pedantic -Wall -Wextra -Werror=return-type -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wnon-virtual-dtor -Wsuggest-override -O3 -DNDEBUG -fPIC -pthread -std=gnu++17 -MD -MT Gen/LbPythia8/CMakeFiles/LbPythia8Lib.dir/src/Lib/Pythia8Production.cpp.o -MF Gen/LbPythia8/CMakeFiles/LbPythia8Lib.dir/src/Lib/Pythia8Production.cpp.o.d -o Gen/LbPythia8/CMakeFiles/LbPythia8Lib.dir/src/Lib/Pythia8Production.cpp.o -c ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp In file included from ../Gen/LbPythia8/include/LbPythia8/GaudiRandomForPythia8.h:16, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:15, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Basics.h: In member function 'std::pair<double, double> Pythia8::Rndm::gauss2()': /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Basics.h:411:33: note: parameter passing for argument of type 'std::pair<double, double>' when C++17 is enabled changed to match C++14 in GCC 10.1 411 | pair<double, double> gauss2() {double r = sqrt(-2. * log(flat())); | ^ ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp: In member function 'virtual StatusCode Pythia8Production::initializeGenerator()': ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:140:29: error: cannot convert 'BeamToolForPythia8*' to 'Pythia8::BeamShapePtr' {aka 'std::shared_ptr<Pythia8::BeamShape>'} 140 | m_pythia->setBeamShapePtr(m_pythiaBeamTool); | ^~~~~~~~~~~~~~~~ | | | BeamToolForPythia8* In file included from ../Gen/LbPythia8/include/LbPythia8/LhcbHooks.h:14, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:17, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Pythia.h:162:38: note: initializing argument 1 of 'bool Pythia8::Pythia::setBeamShapePtr(Pythia8::BeamShapePtr)' 162 | bool setBeamShapePtr( BeamShapePtr beamShapePtrIn) | ~~~~~~~~~~~~~^~~~~~~~~~~~~~ ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:142:31: error: cannot convert 'Pythia8::UserHooks* const' to 'Pythia8::UserHooksPtr' {aka 'std::shared_ptr<Pythia8::UserHooks>'} 142 | m_pythia->addUserHooksPtr(hook); | ^~~~ | | | Pythia8::UserHooks* const In file included from ../Gen/LbPythia8/include/LbPythia8/LhcbHooks.h:14, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:17, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Pythia.h:144:38: note: initializing argument 1 of 'bool Pythia8::Pythia::addUserHooksPtr(Pythia8::UserHooksPtr)' 144 | bool addUserHooksPtr( UserHooksPtr userHooksPtrIn) { | ~~~~~~~~~~~~~^~~~~~~~~~~~~~ ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:144:38: error: cannot convert 'Pythia8::LHAup*' to 'Pythia8::LHAupPtr' {aka 'std::shared_ptr<Pythia8::LHAup>'} 144 | if (m_lhaup) m_pythia->setLHAupPtr(m_lhaup); | ^~~~~~~ | | | Pythia8::LHAup* In file included from ../Gen/LbPythia8/include/LbPythia8/LhcbHooks.h:14, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:17, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Pythia.h:127:30: note: initializing argument 1 of 'bool Pythia8::Pythia::setLHAupPtr(Pythia8::LHAupPtr)' 127 | bool setLHAupPtr( LHAupPtr lhaUpPtrIn) {lhaUpPtr = lhaUpPtrIn; | ~~~~~~~~~^~~~~~~~~~ ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:210:50: error: cannot convert 'Pythia8::ParticleDataEntryPtr' {aka 'std::shared_ptr<Pythia8::ParticleDataEntry>'} to 'Pythia8::ParticleDataEntry*' in initialization 210 | m_pythia->particleData.particleDataEntryPtr(*id); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~ | | | Pythia8::ParticleDataEntryPtr {aka std::shared_ptr<Pythia8::ParticleDataEntry>} ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp: In member function 'virtual StatusCode Pythia8Production::finalize()': ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:247:52: error: passing 'const Pythia8::Info' as 'this' argument discards qualifiers [-fpermissive] 247 | std::vector<int> codes = m_pythia->info.codesHard(); | ~~~~~~~~~~~~~~~~~~~~~~~~^~ In file included from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Settings.h:20, from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/BeamShape.h:13, from ../Gen/LbPythia8/include/LbPythia8/BeamToolForPythia8.h:17, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:16, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Info.h:289:15: note: in call to 'std::vector<int> Pythia8::Info::codesHard()' 289 | vector<int> codesHard(); | ^~~~~~~~~ ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp: In member function 'virtual StatusCode Pythia8Production::generateEvent(HepMC::GenEvent*, LHCb::GenCollision*)': ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:285:52: error: passing 'const Pythia8::Info' as 'this' argument discards qualifiers [-fpermissive] 285 | std::vector<int> codes = m_pythia->info.codesHard(); | ~~~~~~~~~~~~~~~~~~~~~~~~^~ In file included from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Settings.h:20, from /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/BeamShape.h:13, from ../Gen/LbPythia8/include/LbPythia8/BeamToolForPythia8.h:17, from ../Gen/LbPythia8/include/LbPythia8/Pythia8Production.h:16, from ../Gen/LbPythia8/src/Lib/Pythia8Production.cpp:31: /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-fcff2/aarch64-centos7-gcc11-opt/include/Pythia8/Info.h:289:15: note: in call to 'std::vector<int> Pythia8::Info::codesHard()' 289 | vector<int> codesHard(); | ^~~~~~~~~And then a similar error that is also related to -fpermissive
I will try to have a look at these five issues
- Add
Hi Adam, thanks! I tried the following. It seems to have worked (the build error from the missing CaloUtils has disappeared) !914 (e95643bc)
No, you are using
sim10-patches, or you should be usingsim10-patchesEdited by Gloria CortiAh! Thanks Gloria :-) I am NOT using sim10-patches anywhere, but I should be using sim10-patches, then this makes sense.
Is this a global tag I can pass to the hackathon setup? I am using Marco's instructions here https://codimd.web.cern.ch/s/G2qnvDjev#
(I mean I guess I need sim10-patches for Gauss, I guess LHCb, and which other packages? I have Gaudi, GitCondDB, Detector, LHCb, Run2Support. Geant4, Gauss... plus a few others with tags pre-fixed by Marco, like lcg-toolchains, Catch2, yaml-cpp, DD4HEP).
Edited by Andrea ValassiHi again, I had a look at the nightlies in https://lhcb-nightlies.web.cern.ch/nightly/ . I saw there are various sim10 builds. I assume I can work close to this one, https://lhcb-nightlies.web.cern.ch/nightly/lhcb-sim10-gaudi-head/129/ ? This one has
- DBASE: None
- LCG: 101x
- Geant4: v10r6p2t6
- Gaudi: master
- Detector: v1r2
- LHCb: sim10-patches
- Run2Support: HEAD
- Gauss: HEAD
I think that I can try the above, i.e. essentially fix Geant4 and use the LHCb branch sim10-patches as mentioned by Gloria. However I think I also need an aarch64 patch in Detector, so I will make my branch off v1r2 and add Marco's patch.
In summary this is what I suggest that I should try:
- (DBASE: not needed, I have not downloaded this)
- LCG: 102b (the aarch version I am using is /cvmfs/sft.cern.ch/lcg/views/LCG_102b/aarch64-centos7-gcc11-opt)
- Geant4: v10r6p2t6 (at least, I will try this...)
- Gaudi: master
- Detector: v1r2 (or better my branch off v1r2, including Marco's patches which so far have only been merged to master)
- LHCb: sim10-patches (or better my branch off sim10-patches, including Marco's patches which so far have only been merged to master)
- Run2Support: master
- Gauss: master (or better my branch off master, including my patches)
Does this make sense? Thanks
I am actually using Gaudi master, GitCondDB master and Detector as-is v1r2 (it builds out of the box).
I created one branch off sim10-patches for LHCb and I created a MR with a minimal set of patches to build it, LHCb!3921 (merged).
Can you please have a look? Thanks!
Ok the AmpGen issue seems to be that tag 2.3 contains a nicely hardcoded x86_64 here https://gitlab.cern.ch/ampgen/ampgen/-/blob/v2.3/CMakeLists.txt#L226
target_compile_options(AmpGen PUBLIC -march=x86-64)This is in a catch-all "else" for no-SIMD. In the latest master this is changed to '-march=native' (however there are also a few added hooks to disable SIMD! actually native would normaly give the best SIMD available...)
Edited by Andrea Valassimentioned in merge request LHCb!3921 (merged)
As suggested by @gcorti (thanks!) I restarted this work based on LHCb sim10. The full list of packages I use is above in a comment.
I have created an LHCb MR against the sim10-patches branch, LHCb!3921 (merged)
I have closed the Gauss MR !914 (closed) and replaced it by !917 (closed). This is still in WIP and incomplete (it does not fully build yet).
mentioned in merge request !917 (closed)
I made progress on various fronts (and ased also @philten for help on Pythia), but it seems like I am totally stuck on Photos. In summary, it looks like the latest Gauss v56r2, while built on some LCG that probably uses photos 3.64, is using photos 3.56. The CxxInterface and Fortran libraries of Photos, which are needed in Gauss LbEvtGen, are only available in photos 3.56 and not in 3.64. The ARM build of photos 3.56 does not exist, there is only a build of photos 3.64. The options are I imagine:
- EITHER ask someone (who? SFT?) to build photos 3.56 on ARM, and then percolate this though the various LHCb cvmfs installations/mirrrors and cmake lcg-toolchain fragments
- OR disable completely photos in the Gauss build on ARM
What I had done so far, just to get the configuration going, was indeed to disable photos lookup in Gauss cmake. But now there are bits of Gauss code LbEvtGen that need photos, so I would need to disable those in an even harder way.
Can someone please suggest which option we need? Thanks!
PS Just for my own reference, this was my reverse engineering
for f in /cvmfs/lhcb.cern.ch/lib/lhcb/GAUSS/GAUSS_v56r2/InstallArea/x86_64_v2-centos7-gcc11-opt/lib/*so; do if grep -i photos $f; then ldd $f |& grep -i photos | grep -v CXX; echo; fi; done Binary file /cvmfs/lhcb.cern.ch/lib/lhcb/GAUSS/GAUSS_v56r2/InstallArea/x86_64_v2-centos7-gcc11-opt/lib/libEvtGenExternal.so matches libPhotosCxxInterface.so.0 => /cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/photos++/3.56.lhcb1-9cb9b/x86_64-centos7-gcc11-opt/lib/libPhotosCxxInterface.so.0 (0x00007f771e1e2000) libPhotosFortran.so.0 => /cvmfs/lhcb.cern.ch/lib/lcg/releases/MCGenerators/photos++/3.56.lhcb1-9cb9b/x86_64-centos7-gcc11-opt/lib/libPhotosFortran.so.0 (0x00007f771d911000) Binary file /cvmfs/lhcb.cern.ch/lib/lhcb/GAUSS/GAUSS_v56r2/InstallArea/x86_64_v2-centos7-gcc11-opt/lib/libEvtGen.so matches Binary file /cvmfs/lhcb.cern.ch/lib/lhcb/GAUSS/GAUSS_v56r2/InstallArea/x86_64_v2-centos7-gcc11-opt/lib/libLbEvtGenLib.so matches libPhotosCxxInterface.so.0 => not found libPhotosFortran.so.0 => not found Binary file /cvmfs/lhcb.cern.ch/lib/lhcb/GAUSS/GAUSS_v56r2/InstallArea/x86_64_v2-centos7-gcc11-opt/lib/libLbEvtGen.so matches ...And on the ARM build node
[root@lbbuildinter02 workspace]# grep 3\.56\.lhcb . -r ... ./lcg-toolchains/fragments/packages/LCG_101_LHCB_7-x86_64-centos7-gcc11-opt.cmake:_add_lcg_entry("${LCG_releases_base}/MCGenerators/photos++/3.56.lhcb1-9cb9b/x86_64-centos7-gcc11-opt") ...In SFT cvmfs LCG101 normally uses 3.64, but many 3.56 builds exist also for lhcb (none for aarch64)
[avalassi@aicoral71 bash] ~> ls /cvmfs/sft.cern.ch/lcg/releases/LCG_101/MCGenerators/photos++/ ./ ../ 3.64/ [avalassi@aicoral71 bash] ~> ls /cvmfs/sft.cern.ch/lcg/releases/MCGenerators/photos++/ ./ 3.61-0f205/ 3.64-22800/ 3.64-8b6f1/ 3.64-c67c6/ ../ 3.61-1e0a6/ 3.64-26eda/ 3.64-8d45e/ 3.64-c8e36/ 3.52-6aaaa/ 3.61-471da/ 3.64-2d912/ 3.64-929a8/ 3.64-caf72/ 3.56-20331/ 3.61-4961c/ 3.64-3252b/ 3.64-9560e/ 3.64-cb374/ 3.56-768ab/ 3.61-5dfcc/ 3.64-3af42/ 3.64-96697/ 3.64-cdabc/ 3.56-8dfe5/ 3.61-78b6c/ 3.64-3c610/ 3.64-96cb8/ 3.64-d33d7/ 3.56-8fc8d/ 3.61-95f9d/ 3.64-3f2e2/ 3.64-9f16b/ 3.64-d54a1/ 3.56-933ef/ 3.61-a18b9/ 3.64-400c1/ 3.64-9f4af/ 3.64-d63f0/ 3.56-e61d6/ 3.61-a5f77/ 3.64-4f910/ 3.64-a1b8c/ 3.64-d8fd2/ 3.56.lhcb1-0fe95/ 3.61-bf7ba/ 3.64-5b9df/ 3.64.atlas-42d4d/ 3.64-dcc67/ 3.56.lhcb1-18dda/ 3.61-c2600/ 3.64-625cf/ 3.64.atlas-4da01/ 3.64-de62d/ 3.56.lhcb1-220a4/ 3.61-cbcf2/ 3.64-6341a/ 3.64.atlas-892a6/ 3.64-e006e/ 3.56.lhcb1-25a11/ 3.61-f26a1/ 3.64-650a0/ 3.64.atlas-b0453/ 3.64-e715c/ 3.56.lhcb1-50520/ 3.61.lhcb-5e533/ 3.64-67cca/ 3.64.atlas-ceec7/ 3.64-e98a4/ 3.56.lhcb1-6ddab/ 3.64-05ee9/ 3.64-67e7a/ 3.64.atlas-f7db7/ 3.64-eacb0/ 3.56.lhcb1-6e05d/ 3.64-06e4a/ 3.64-6991e/ 3.64-b6528/ 3.64-ed9e2/ 3.56.lhcb1-9cb9b/ 3.64-0a562/ 3.64-72ab8/ 3.64-bbb74/ 3.64-f2f65/ 3.56.lhcb1-a9eb5/ 3.64-1a882/ 3.64-7c83d/ 3.64-bdd65/ 3.64-faf21/ 3.56.lhcb1-e52d0/ 3.64-1b8b6/ 3.64-80abc/ 3.64-c3ec1/ 3.64-fbc47/ 3.60-7f718/ 3.64-1c8ea/ 3.64-80bb7/ 3.64-c4464/ 3.64-fdba6/ 3.61-0c409/ 3.64-1fdc2/ 3.64-8173c/ 3.64-c5d09/ .cvmfscatalogEdited by Andrea Valassi-
Let me discuss this with @clemenci and will get back to you
Short update: I managed to get to the end, now make Gauss completes successfully.
With many limitations of course! The main one is that I completely disabled Photos. We need 3.56 and this is missing for ARM. So I simply disabled some build units, or worse I disabled the photos code calls in LbPhotos.cpp and similar.
The last item to debug was also the most bizarre: in my podman environment the USER env is not set... or maybe getenv does not work well. I changed one 'getenv("USER")' in AmpGen to 'AmpGen" in the 'Generated by' headers.
Let's iterate the discussion offline. Thanks to all for the help!
mentioned in merge request !920 (merged)
Hi @gcorti thanks. The ARM stack for LCG102b_LHCB_7 has now been built thanks to EP-SFT, and the configuration has been created thankd to @bcouturi and @clemenci.
I have restarted all of the machinery above. Using the new stack, the Gauss patches that are needed are much simpler, essentially there is only two minor patches for AmpGen, !920 (merged) (I also reported one AmpGen issue upstream as https://github.com/GooFit/AmpGen/issues/30)
I confirm that
make Gaussnow builds succesfully for ARM. I have not tried any test yet however (and I do not really plan to as I am not sure I would be of much help there).I am posting below (in the next comment) the full instructions I am using. Note anyway that this is what I use as branches:
cd Gaudi; git checkout master; cd - cd GitCondDB; git checkout master; cd - cd Detector; git checkout v1r2; cd - cd LHCb; git checkout valassi_sim10_aarch64; cd - cd Run2Support; git checkout master; cd - cd Geant4; git checkout v10r6p2t6; cd - cd Gauss; git checkout valassi_sim10_aarch64_lhcb7; cd -Essentially, only these two MRs are still missing
- In LHCb, merge valassi_sim10_aarch64 into sim10-patches: LHCb!3921 (merged)
- In Gauss, merge valassi_sim10_aarch64_lhcb7 into master: !920 (merged)
Edited by Andrea ValassiThis is the full list of what I am doing, based on Marco's https://codimd.web.cern.ch/Urg1_B6RQPeXRNQUpiVtGQ?view#
Login ssh andreav@lbbuildinter02 bash Check this is ok from .bashrc: env | grep XDG XDG_DATA_HOME=/home/andreav/.local/share XDG_CONFIG_HOME=/home/andreav/.config ... Check this is ok too podman login gitlab-registry.cern.ch Start as per Marco's recipe, WITHOUT SPECIFYING LHCB_7 HERE! cd /home/andreav/ARM2022 git clone https://gitlab.cern.ch/lhcb/upgrade-hackathon-setup.git workspace echo platform=armv8.1_a-centos7-gcc11-opt >> workspace/configuration.mk echo LCG_VERSION=102b >> workspace/configuration.mk echo WITH_GITCONDDB = 1 >> workspace/configuration.mk And add my own patch echo PROJECTS += GitCondDB >> workspace/configuration.mk To speed up the builds, copy the ccache from a previous build cp -dpr workspace.20230117b/.ccache/ workspace Start the container with the tag! This is a new container that includes the latest HEP_OSlibs cd /home/andreav/ARM2022 podman run -ti --network host -v /cvmfs:/cvmfs:shared -v ${PWD}/workspace:/workspace:z -w /workspace gitlab-registry.cern.ch/lhcb-core/lbdocker/aarch64-centos7-build:2023-01-05 Everything else is within the container. Then within the container as per Marco's instructions, WITHOUT SPECIFYING LHCB7! . /cvmfs/lhcb.cern.ch/lib/LbEnv lb-set-platform armv8.1_a-centos7-gcc11-opt export LCG_VERSION=102b git clone -b add-aarch64 https://gitlab.cern.ch/lhcb-core/lcg-toolchains.git git clone https://gitlab.cern.ch/lhcb-core/mirrors/Catch2.git -b v2.13.10 cmake -S Catch2 -B Catch2/build.$BINARY_TAG -DCMAKE_TOOLCHAIN_FILE=${PWD}/lcg-toolchains/LCG_${LCG_VERSION}/$BINARY_TAG.cmake -DCATCH_ENABLE_WERROR=NO -DCATCH_BUILD_STATIC_LIBRARY=YES -DCMAKE_INSTALL_PREFIX=${PWD}/Catch2/InstallArea/$BINARY_TAG -DBUILD_TESTING=NO -GNinja cmake --build Catch2/build.$BINARY_TAG --target install git clone https://gitlab.cern.ch/lhcb-core/mirrors/yaml-cpp.git -b yaml-cpp-0.7.0 cmake -S yaml-cpp -B yaml-cpp/build.$BINARY_TAG -DCMAKE_TOOLCHAIN_FILE=${PWD}/lcg-toolchains/LCG_${LCG_VERSION}/$BINARY_TAG.cmake -DCMAKE_INSTALL_PREFIX=${PWD}/yaml-cpp/InstallArea/$BINARY_TAG -DYAML_BUILD_SHARED_LIBS=ON -DBUILD_TESTING=NO -GNinja cmake --build yaml-cpp/build.$BINARY_TAG --target install git clone https://gitlab.cern.ch/lhcb-core/mirrors/DD4hep.git -b v01-23 ( . /cvmfs/sft.cern.ch/lcg/views/LCG_${LCG_VERSION}/aarch64-centos7-gcc11-opt/setup.sh cmake -S DD4hep -B DD4hep/build.$BINARY_TAG -DCMAKE_INSTALL_PREFIX=${PWD}/DD4hep/InstallArea/$BINARY_TAG -DBUILD_TESTING=NO -GNinja -DCMAKE_CXX_STANDARD=17 -DDD4HEP_USE_XERCESC=ON -DDD4HEP_USE_GEANT4=OFF -DDD4HEP_USE_TBB=ON -DDD4HEP_BUILD_PACKAGES="DDRec DDDetectors DDCond DDAlign" cmake --build DD4hep/build.$BINARY_TAG --target install ) Download the projects git clone ssh://git@gitlab.cern.ch:7999/gaudi/Gaudi.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/GitCondDB.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/Detector.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/LHCb.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/Run2Support.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/Geant4.git git clone ssh://git@gitlab.cern.ch:7999/lhcb/Gauss.git Fix the branches or tags in all packages cd Gaudi; git checkout master; cd - cd GitCondDB; git checkout master; cd - cd Detector; git checkout v1r2; cd - cd LHCb; git checkout valassi_sim10_aarch64; cd - cd Run2Support; git checkout master; cd - cd Geant4; git checkout v10r6p2t6; cd - cd Gauss; git checkout valassi_sim10_aarch64_lhcb7; cd - Build all packages make Gaudi make GitCondDB make Detector make LHCb make Run2Support make Geant4 make GaussWith LCG_102b you do not need anymore the GitCondDB part because it's included in the LCG build.
Your recipe can now be reduced to something like:
- preparation
git clone https://gitlab.cern.ch/lhcb/upgrade-hackathon-setup.git workspace cat >> workspace/configuration.mk <<EOF platform=armv8.1_a-centos7-gcc11-opt LCG_VERSION=102b Detector_BRANCH=v1r2 LHCb_BRANCH=valassi_sim10_aarch64 Geant4_BRANCH=v10r6p2t6 Gauss_BRANCH=valassi_sim10_aarch64_lhcb7 EOF - start the container
podman run -ti --network host -v /cvmfs:/cvmfs:shared -v ${PWD}/workspace:/workspace:z -w /workspace gitlab-registry.cern.ch/lhcb-core/lbdocker/aarch64-centos7-build:2023-01-05 - build (in the container)
. /cvmfs/lhcb.cern.ch/lib/LbEnv make Gauss
Edited by Marco Clemencic- preparation
mentioned in issue Geant4#10 (closed)
One more general comment. Using Marco's latest instructions above I have rebuilt the stack for ARM. Then I also tested it using the same HEP-SCORE setup that I used to evaluate the Geant 10.7 patch in Geant4#10 (closed)
cd Gauss wget https://gitlab.cern.ch/valassi/hep-workloads/-/raw/qa-build-lhcb-sim-run3/lhcb/sim-run3/lhcb-sim-run3/prodConf_Gauss_0bmk2023_00000726_1.py cat prodConf_Gauss_0bmk2023_00000726_1.py | sed -e "s/NOfEvents=5/NOfEvents=5/" > prodConf005evt.py PRODCONFROOT=/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/ProdConf/v3r5 PYTHONPATH=/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/ProdConf/v3r5/python ./run gaudirun.py -T '$APPCONFIGOPTS/Gauss/Beam6800GeV-mu100-2022-nu3.2.py' '$DECFILESROOT/options/10000000.py' '$LBPYTHIA8ROOT/options/Pythia8.py' '$APPCONFIGOPTS/Gauss/Run3-detector.py' '$APPCONFIGOPTS/Gauss/DataType-Upgrade.py' '$APPCONFIGOPTS/Gauss/G4PL_FTFP_BERT_EmOpt2.py' '$APPCONFIGOPTS/Persistency/Compression-ZLIB-1.py' prodConf005evt.py 2>&1 | tee out005evt_OLD_arm.logThis completes successfully. However the results are not bit by bit the same as for the same instructions on x86. If I compare the two logs with 5 events, this is what I get
[root@lbbuildinter02 Gauss]# python -c "from parseResults import *; parseLogTxt('out005evt_OLD_x86.log')" | \grep ^iCPU | grep -v GEN iCPU=xx [ time, evt, #part, ##, #vert, ##, G/S] time/k#part iCPU=00 [ 81.806, 1001, 5224, 1, 5653, 1, SIM] 15.65965 iCPU=00 [ 34.668, 1002, 3329, 1, 3519, 1, SIM] 10.41394 iCPU=00 [ 70.166, 1003, 4904, 1, 5221, 1, SIM] 14.30791 iCPU=00 [ 36.506, 1004, 3199, 1, 3463, 1, SIM] 11.41169 iCPU=00 [ 37.402, 1005, 2480, 1, 2726, 1, SIM] 15.08145 [root@lbbuildinter02 Gauss]# python -c "from parseResults import *; parseLogTxt('out005evt_OLD_arm.log')" | \grep ^iCPU | grep -v GEN iCPU=xx [ time, evt, #part, ##, #vert, ##, G/S] time/k#part iCPU=00 [ 113.857, 1001, 5464, 1, 5949, 1, SIM] 20.83766 iCPU=00 [ 52.072, 1002, 3246, 1, 3456, 1, SIM] 16.04190 iCPU=00 [ 79.965, 1003, 4819, 1, 4991, 1, SIM] 16.59369 iCPU=00 [ 54.483, 1004, 3709, 1, 3978, 1, SIM] 14.68940 iCPU=00 [ 52.577, 1005, 2453, 1, 2750, 1, SIM] 21.43375In other words the "GiGaGetMainEvent INFO Number of extracted MCParticles/MCVertices" printouts are different. The numbers differ so much that I am tempted to imagine that different sets of random numbers are used somewhere in the algorithms... otherwise there is really something badly wrong/different.
Questions for @clemenci :
- Is it possible to print out all random numbers used everywhere?
- Is it possible that different random numbers are used on x86 and arm, and why? Can I force them to be the same?
- Vague idea, could it be that using x86-64_v2 switches on a vectorization of random numbers that results in different sequences? (I will try by rebuilding without the v2 just in case...) Thanks Andrea
Edited by Andrea Valassi-
@valassi, just to clarify, are you running on x86 the exact same code - caveat the unavoidable differences for arms?
I am not so surprised that over a run of 5 events you see differences: even if the random numbers seed are the same the sequence is accessing registers in memory. We do see sometimes differences in the event evolution between opt and dbg. The memory layout can be different and as soon a different register is access because of the randomisation of a physics process this get reflected in what happens later and the particles we decide to keep in MCParticles may change.
Do do a fair comparison we need to run a few thousands events and compare them statistically.
@gcorti, yes unless I am doing something wrong (always a possibility!) I am running the exact same git tags of all projects for x86 and ARM. Ok if you are not surprised that there are differences, then this is not necessarily a bad sign. I agree with you that the full validation should be done statistically on thousands of events: @landerli had kindly suggested he might be able to help with this, but not necessarily on a short timescale.
Would you agree that we could still aim for a release on cvmfs (both x86 and ARM) and the inclusion in HEP-SCORE by mid-February of these ARM patches, even if the full physics validation has not been done yet? I agree that the ARM version would not be usable by LHC in production yet, but from the point of view of HEP-SCORE benchmarking I think that it's better to have an ARM-to-be-validated version rather than no ARM version at all.
Thanks Andrea
