Start versioning the CI reference files for the RunTier0Tests
This is an important merge request that'll (eventually) affect a number of CI tests. Currently we run qTestsTier0_required-test
in 21.0 and 21.3 branches, SimulationTier0Test_required-test
in 21.0, 21.3, 21.9, and master, and OverlayTier0Test_required-test
in 21.0 and master. Current short coming of the tests is that the reference files are created (automatically or manually) based on the nightlies. When a MR which changes the relevant outputs is accepted all CI builds that follow until the reference file(s) is(are) updated report errors. This apparently caused a lot of confusion to the release shifters recently as pointed out in ATEAM-480.
With this MR we're starting to version the reference files that'll allow keeping multiple version at a time. In the new scheme, when a MR comes that changes any of the relevant files, the following steps will have to be taken:
- The MR is submitted as usual, the relevant test failing
- The developer creates a new reference file locally
- Once the changes are approved an "expert" (can be defined by the coordinators) will create a subfolder under
/eos/atlas/atlascerngroupdisk/data-art/grid-input/Tier0ChainTests/test/branch
, i.e. to createv1
for theq221
in21.0
this means/eos/atlas/atlascerngroupdisk/data-art/grid-input/Tier0ChainTests/q221/21.0/v1
(we can change the versioning system at anytime) and copy the file there (will need write access obviously) - The developer updates the MR by updating mapping in
RunTier0TestsTools.py
to point to the new version - The CI build re-runs (picking the new version) and succeeds
- Once the MR goes in all subsequent CI builds will start using the new file
Please take a note of the change. With this we're going to drop automatically producing the reference files as it's not needed anymore.
I'm cc-ing the relevant people I can think of: @janders @jbeacham @jchapman @tadej @fwinkl