Skip to content

make images grid compatible

Lukas Alexander Heinrich requested to merge makeGridCompatible into master

This is a somewhat ugly MR. The issue is the following.

For container runtimes that use an overlay filesystem (like Docker) when you call

docker run -v /some/path:/some/nonexisting/path <image> <cmd>

the system is smart enough to create the mountpoint in the root filesystem of the container if it does not exist. But when using singularity as a runtime without overlay, it cannot do that.

This MR adds /alrb, /cvmfs as mount points since these are expected for running containers on the grid. I'm not super happy about this since our images neither need ALRB nor CVMFS but after a very long discussion in https://its.cern.ch/jira/browse/ADCINFR-83 I think this is a acceptable short-term solution (in the end it's just two directories being created)

the issue is that on the grid the current plan is to construct the right singularity command via a call to setupATLAS

e.g. sth like setupATLAS -c docker://lukasheinrich/someimage

which will actually then run

singularity  exec  -e  -H /some/alrbdir:/alrb -B /some/homedir:/home -B /some/workdir:/srv -B /cvmfs:/cvmfs docker://lukasheinrich/someimage /bin/bash

of course we do not need many of those bind mounts in fact we would be perfectly content with

singularity  exec  -e  -H /some/homedir:/home -B /some/workdir:/srv docker://lukasheinrich/someimage /bin/bash

but at this point those additional bindmounts do not hurt and in turn we gain the ability to run containers on the grid (via the soon-to-come --containerImage flag in prun and people can easily start they container images on lxplus via

setupATLAS -q -c docker://gitlab.cern.ch/myproject/myimage

I added a note that this should be temporary and removed as soon as we don't need it

pinging @mavogel, @akraszna

Edited by Attila Krasznahorkay

Merge request reports