This is an important merge request that'll (eventually) affect a number of CI tests. Currently we run
qTestsTier0_required-test in 21.0 and 21.3 branches,
SimulationTier0Test_required-test in 21.0, 21.3, 21.9, and master, and
OverlayTier0Test_required-test in 21.0 and master. Current short coming of the tests is that the reference files are created (automatically or manually) based on the nightlies. When a MR which changes the relevant outputs is accepted all CI builds that follow until the reference file(s) is(are) updated report errors. This apparently caused a lot of confusion to the release shifters recently as pointed out in ATEAM-480.
With this MR we're starting to version the reference files that'll allow keeping multiple version at a time. In the new scheme, when a MR comes that changes any of the relevant files, the following steps will have to be taken:
- The MR is submitted as usual, the relevant test failing
- The developer creates a new reference file locally
- Once the changes are approved an "expert" (can be defined by the coordinators) will create a subfolder under
/eos/atlas/atlascerngroupdisk/data-art/grid-input/Tier0ChainTests/test/branch, i.e. to create
/eos/atlas/atlascerngroupdisk/data-art/grid-input/Tier0ChainTests/q221/21.0/v1(we can change the versioning system at anytime) and copy the file there (will need write access obviously)
- The developer updates the MR by updating mapping in
RunTier0TestsTools.pyto point to the new version
- The CI build re-runs (picking the new version) and succeeds
- Once the MR goes in all subsequent CI builds will start using the new file
Please take a note of the change. With this we're going to drop automatically producing the reference files as it's not needed anymore.
I'm cc-ing the relevant people I can think of: @janders @jbeacham @jchapman @tadej @fwinkl