Disable CUDA support in AthAnalysis, master branch (2022.07.07.)

Turned off CUDA support for ONNXRuntime in AthAnalysis. Following up from !54769 (merged).

As @gouighri noticed while building the AthAnalysis-22.2.81 release, it became a whole lot bigger than AthAnalysis-22.2.80. Which may be okay on CVMFS, but it's not okay for the Docker container.

https://hub.docker.com/repository/docker/atlas/athanalysis/tags?page=1&ordering=last_updated

This is because of:

[bash][atspot01]:~ > docker run -it --rm atlas/athanalysis:22.2.81
             _ _____ _      _   ___ 
            /_\_   _| |    /_\ / __|
           / _ \| | | |__ / _ \\__ \
          /_/ \_\_| |____/_/ \_\___/

This is a self-contained ATLAS AthAnalysis image.
To set up the analysis release of the image, please
execute:

          source /release_setup.sh

[bash][atlas]:workdir > du -sh /opt/lcg/* | sort -h
44K     /opt/lcg/LCG_101_ATLAS_24
92K     /opt/lcg/pytest_runner
...
579M    /opt/lcg/ROOT
612M    /opt/lcg/mysql
635M    /opt/lcg/oracle
5.7G    /opt/lcg/cuda
8.8G    /opt/lcg/cudnn
[bash][atlas]:workdir >

It would've been cool to have CUDA support for ML like this by default in AthAnalysis, but it really just makes these containers unusable. @dbakshig, I'm still curious what you think. But I don't think we can do anything other than this (for now at least).

assigned to @akraszna

This merge request affects 1 package:

• Projects/AthAnalysis

This merge request affects 1 file:

• Projects/AthAnalysis/build_externals.sh

Adding @krumnack ,@akraszna as watchers

added Build master review-pending-level-1 labels

Hi @akraszna ,

If the memory increase is more because of cudnn (8.8G) we can patch this part in https://github.com/microsoft/onnxruntime/blob/master/cmake/CMakeLists.txt#L1001

set(onnxruntime_LINK_DIRS )
if (onnxruntime_USE_CUDA)
      #TODO: combine onnxruntime_CUDNN_HOME and onnxruntime_CUDA_HOME, assume they are the same
      if (WIN32)
        list(APPEND onnxruntime_LINK_DIRS ${onnxruntime_CUDNN_HOME}/lib/x64 ${onnxruntime_CUDA_HOME}/x64/lib64)
      else()
        list(APPEND onnxruntime_LINK_DIRS ${onnxruntime_CUDNN_HOME}/lib64 ${onnxruntime_CUDA_HOME}/lib64)
      endif()
endif()

Let me know what you think.

Thanks, Debo.

Hi Debo,

The issue is a little different. Just by calling

find_package( CUDA )
find_package( cuDNN )

during the build of AthAnalysisExternals (and those packages actually being found), the analysis release RPM is set up to depend on the CUDA and cuDNN RPMs. So there's just no way of enabling CUDA support for ONNXRuntime without making the release this much bigger.

Since the CUDA support is introduced through a "plugin library", we could think of ways of providing an atlas/athanalysis-cuda image that would add the plugin library, with all of its dependencies. So that if users ask for CUDA support with their analysis release, they could get it. But the default image can not afford to have all this much stuff in it.

Cheers, Attila

Hi @akraszna ,

Ok understood so nothing we can do by fixing onnxruntime.

Thanks, Debo.

CI Result SUCCESS (hash 4d05fa5e)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 55411]

removed review-pending-level-1 label

added review-approved label

merged

mentioned in commit 7e1923a6

added sweep:ignore label