To include additional files to be sent to grid (e.g. some custom calibration files) with the
prun command, the desired files (or directories) needs to be in the job submission directory (and cannot be somewhere else).
PrunDriver the submission occurs in
jobELGDir = data.submitDir + "/elg"
Hence to be able to send additional files or directories to grid they need to be in the
This MR enables to do so by creating symbolic links of files (or directories) that the user wants to include.
It is done thanks to the use of a new flag
which should contain the list of path to be included separated with comma
NB: for heavy files one needs also to ask for
prun not to ignore them by using
EL::Job::optUserFiles and providing only the name of files (not the path), also comma separated.
E.g. for our group we want to send a custom CDI file (stored on EOS) we need to call in our submission script
// for symbolic link creation in the submission directory driver->options()->setString( EL::Job::optGridPrunShipAdditionalFilesOrDirs, "/eos/atlas/atlascerngroupdisk/phys-higgs/HSG5/Run2/VH/bTaggingCDI/2020-21-13TeV-MC16-CDI_smooth_VHLegacy-07Oct-v02.root") ; // for prun to not skip that heavy CDI file in the submission directory driver->options()->setString( EL::Job::optUserFiles, "2020-21-13TeV-MC16-CDI_smooth_VHLegacy-07Oct-v02.root");
I tested that the CDI file is correctly copied in the tar.gz file sent to the grid.