The source project of this merge request has been removed.
Implement new CI job that checks GRID file accessibility and size
This MR implements a new CI job that will run scripts/check_grid_files.sh
and will test:
- whether the directory in which the file lives on eos is readable by
atlcvmfs
. If it's not the job will fail - the job will also fail if the file is not stored on
/eos/atlas
or/eos/user
- the job will fail if the files uploaded are bigger than 100 MB
The checks can be skipped using [skip eos]
.
One thing to discuss would be whether we want to split this job into 2:
- checking the accessibility by
atlcvmfs
- checking the size
since we might want to skip the size check but not the
atlcvmfs
readability check. @fsiegert @amoroso @ewelina what do you think?sizes.txt
The average files of existing non-LHE files seems to be 40 MB, while the median is 20 MB. If we want we could reduce the size even further - depending on how much fine tuning we want.
The output of a successful pipeline based on what is in 421xxx/421003
can be found here