Skip to content

Draft: VecMem Based Examples, main branch (2023.12.19.)

Attila Krasznahorkay requested to merge akraszna/athena:VecMemExamples-main-20231219 into main

Following !67871 (merged), here are some examples of how to make use of vecmem in Athena. 😄

  • In AthExCUDA I wrote some code that demonstrates (in a for now fairly awkward way) how data from an xAOD container could be transferred back and forth with a VecMem SoA container;
  • In AthExSYCL I updated the build setup to make use of the CMake code developed for the Acts R&D projects for building SYCL source files.

The first part is meant for @fwinkl and @dossantn primarily at this point. (Though @bwynne should also ping the FastTrackFinder developers here, as I'll also want to get them on board with this...)

The second part of mainly for @tbold, and their group's developments. Note that with the updated recipe that I put into that package's README, I was able to run this MR's code successfully on top of our current x86_64-el9-gcc13-opt nightly. 😄

[bash][pcadp04]:build-workdir > ./CMakeFiles/atlas_build_run.sh athena.py --stdcmalloc AthExSYCL/LinearTransformExample_jobOptions.py
Tue Dec 19 14:50:57 CET 2023
Py:Athena            INFO using release [WorkDir-24.0.20] [x86_64-el9-gcc13-opt] [VecMemExamples-main-20231219/2e45db7f0d4] -- built on [2023-12-19T1444]
Py:Athena            INFO including file "AthenaCommon/Preparation.py"
Py:Athena            INFO including file "AthenaCommon/Atlas.UnixStandardJob.py"
Py:Athena            INFO executing ROOT6Setup
Py:Athena            INFO including file "AthenaCommon/Execution.py"
Py:Athena            INFO including file "AthExSYCL/LinearTransformExample_jobOptions.py"
Py:Athena            INFO including file "AthenaCommon/runbatch.py"
Py:ConfigurableDb    INFO Read module info for 5257 configurables from 4 genConfDb files
Py:ConfigurableDb    INFO No duplicates have been found: that's good !
ApplicationMgr    SUCCESS 
====================================================================================================================================
                                                   Welcome to ApplicationMgr (GaudiCoreSvc v37r2)
                                          running on pcadp04 on Tue Dec 19 14:53:56 2023
====================================================================================================================================
ApplicationMgr       INFO Application Manager Configured successfully
AthDictLoaderSvc     INFO in initialize...
AthDictLoaderSvc     INFO acquired Dso-registry
ClassIDSvc           INFO getRegistryEntries: read 3092 CLIDRegistry entries for module ALL
CoreDumpSvc          INFO Handling signals: 11(Segmentation fault) 7(Bus error) 4(Illegal instruction) 8(Floating point exception) 14(Alarm clock) 
AthenaEventLoopMgr   INFO Initializing AthenaEventLoopMgr
AthSYCL::Linear...  DEBUG Property update for OutputLevel : new value = 2
AthSYCL::Linear...  DEBUG input handles: 0
AthSYCL::Linear...  DEBUG output handles: 0
ApplicationMgr       INFO Application Manager Initialized successfully
ApplicationMgr       INFO Application Manager Started successfully
AthenaEventLoopMgr   INFO   ===>>>  start of run 1    <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #1, run #1 0 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #1, run #1 1 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #2, run #1 1 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #2, run #1 2 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #3, run #1 2 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #3, run #1 3 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #4, run #1 3 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #4, run #1 4 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #5, run #1 4 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #5, run #1 5 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #6, run #1 5 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #6, run #1 6 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #7, run #1 6 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #7, run #1 7 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #8, run #1 7 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #8, run #1 8 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #9, run #1 8 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #9, run #1 9 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #10, run #1 9 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device Intel(R) Data Center GPU Flex 170 (1.3)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #10, run #1 10 events processed so far  <<<===
ApplicationMgr       INFO Application Manager Stopped successfully
AthSYCL::Linear...  DEBUG Calling destructor
AthDictLoaderSvc     INFO in finalize...
ToolSvc              INFO Removing all tools created by ToolSvc
ApplicationMgr       INFO Application Manager Finalized successfully
ApplicationMgr       INFO Application Manager Terminated successfully
Py:Athena            INFO leaving with code 0: "successful run"
[bash][pcadp04]:build-workdir >

The MR is a draft as a start until !67924 (merged) is sorted out, and I can remove the updates I had to put into the branch for testing the CUDA code with, which would fix: ATLASRECTS-7864

Merge request reports