Skip to content

GitLab

Merged
Created by Spyros Argyropoulos@sargyropMaintainer

Implement new CI job that checks GRID file accessibility and size

This MR implements a new CI job that will run scripts/check_grid_files.sh and will test:

  • whether the directory in which the file lives on eos is readable by atlcvmfs. If it's not the job will fail
  • the job will also fail if the file is not stored on /eos/atlas or /eos/user
  • the job will fail if the files uploaded are bigger than 100 MB

The checks can be skipped using [skip eos].

One thing to discuss would be whether we want to split this job into 2:

  1. checking the accessibility by atlcvmfs
  2. checking the size since we might want to skip the size check but not the atlcvmfs readability check. @fsiegert @amoroso @ewelina what do you think?sizes.txt

The average files of existing non-LHE files seems to be 40 MB, while the median is 20 MB. If we want we could reduce the size even further - depending on how much fine tuning we want.

The output of a successful pipeline based on what is in 421xxx/421003 can be found here

sizes.txt sizesh.txt

Edited by Spyros Argyropoulos