AthExSYCL Creation, master branch (2020.03.04.)

😄 I'm quite proud of myself with this one...

After figuring out how to build SYCL code correctly against the current nightlies (more on this later...), I decided to add an example package similar to that of Control/AthenaExamples/AthExCUDA. The code in this package does exactly the same as what AthExCUDA does. It applies a trivial transformation on a small vector of floats.

Just like AthExCUDA, this package should also recuse itself from the build if it finds that the compiler used for the build will not understand its code. Unlike AthExCUDA though this package has to do this "by hand". I wrote a FindSYCL.cmake cmake module for the package, which should later on be put into some more central place, for doing exactly this. Note that the module even knows how to make binaries that contain code for both Intel and NVidia devices at the same time. 😉

Now, for the compilation... There are basically two ways that I know of right now...

1. I've put a build of intel/llvm under /afs/cern.ch/user/k/krasznaa/work/public/clang/11.0.0-ef68270/x86_64-centos7-gcc8-opt. The environment for building this package can be set up using that, like:

setupATLAS
asetup Athena,master,latest
source /afs/cern.ch/work/k/krasznaa/public/clang/11.0.0-ef68270/x86_64-centos7-gcc8-opt/setup.sh

With such a build I can run the example like the following on a private CentOS 7 machine of mine, which has the Intel OpenCL driver installed for the machine's CPU. (The Intel GPU OpenCL drivers don't really work on CentOS 7 unfortunately...)

[bash][pcadp02]:run > athena.py AthExSYCL/LinearTransformExample_jobOptions.py
Wed Mar  4 14:43:10 CET 2020
Preloading tcmalloc_minimal.so
Py:Athena            INFO including file "AthenaCommon/Preparation.py"
Py:Athena            INFO using release [WorkDir-22.0.10] [x86_64-centos7-clang11-opt] [AthExSYCL-master-20200304/0d7cafe] -- built on [2020-03-04T1316]
Py:Athena            INFO including file "AthenaCommon/Atlas.UnixStandardJob.py"
Py:Athena            INFO executing ROOT6Setup
Py:Athena            INFO including file "AthenaCommon/Execution.py"
Py:Athena            INFO including file "AthExSYCL/LinearTransformExample_jobOptions.py"
Py:Athena            INFO including file "AthenaCommon/runbatch.py"
...
AthenaEventLoopMgr   INFO   ===>>>  start of run 1    <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #1, run #1 0 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device         Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (OpenCL 2.1 (Build 0))
AthenaEventLoopMgr   INFO   ===>>>  done processing event #1, run #1 1 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #2, run #1 1 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device         Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (OpenCL 2.1 (Build 0))
AthenaEventLoopMgr   INFO   ===>>>  done processing event #2, run #1 2 events processed so far  <<<===
...

(On lxplus it would just fall back on the "host device".)

2. It's also possible to build the code using the Docker image generated by this repository:

https://gitlab.cern.ch/akraszna/atlas-gpu-devel-env

This is something that I'll be advertising a lot more shortly, but in short, inside the image one can just set up an environment for building both CUDA and SYCL code at the same time with:

setupATLAS
asetup Athena,master,latest

The image has a custom setup for asetup that makes it set up the Intel Clang compiler shipped with the image automatically. 😃 The code then runs like the following on an Ubuntu 18.04 machine:

[bash][atlas]:build > athena.py AthExSYCL/LinearTransformExample_jobOptions.py                      
Wed Mar  4 13:32:23 UTC 2020
Preloading tcmalloc_minimal.so
Py:Athena            INFO including file "AthenaCommon/Preparation.py"
Py:Athena            INFO using release [WorkDir-22.0.10] [x86_64-centos7-clang11-opt] [AthExSYCL-master-20200304/0d7cafe] -- built on [2020-03-04T1329]
Py:Athena            INFO including file "AthenaCommon/Atlas.UnixStandardJob.py"
Py:Athena            INFO executing ROOT6Setup
Py:Athena            INFO including file "AthenaCommon/Execution.py"
Py:Athena            INFO including file "AthExSYCL/LinearTransformExample_jobOptions.py"
Py:Athena            INFO including file "AthenaCommon/runbatch.py"
...
AthenaEventLoopMgr   INFO   ===>>>  start of run 1    <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #1, run #1 0 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device GeForce RTX 2060 (PI 0.0)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #1, run #1 1 events processed so far  <<<===
AthenaEventLoopMgr   INFO   ===>>>  start processing event #2, run #1 1 events processed so far  <<<===
AthSYCL::Linear...  DEBUG Using device GeForce RTX 2060 (PI 0.0)
AthenaEventLoopMgr   INFO   ===>>>  done processing event #2, run #1 2 events processed so far  <<<===
...

As I started, I'm pretty pleased with myself... 😄

Pinging @leggett, @fwinkl, @baines, @ssnyder, @tsulaia.