Major rewrite of TrigUpgradeTest - remove ctests
Apologies for the large MR, but this is a major disruption and needs to be done all in one go. Since many files are renamed and then modified, GitLab diff displays some of them as deleted and new files added.
Description
This MR addresses a long-standing issue of TrigUpgradeTest and the trigger developer workflow abusing the unit test functionality to run large integration tests running athena with tens of components. This is what ART is for. Thus, this MR properly interfaces all TrigUpgradeTest tests to the trigger validation infrastructure and at the same time removes them from the list of unit tests. When this is merged, the tests will be only ART tests. I also provide a new script for developers to run all or selected tests easily and see the summary of the results.
Changes
In more details, this MR includes the following changes:
- Remove nearly all tests from TUT unit tests in CMakeLists.txt. The remaining few are real unit tests, although they test units from other packages and should be moved in subsequent MRs.
- Rename all test scripts to implement new unified naming scheme proposed in ATR-19735.
- Interface all tests to trigger validation infrastructure including common execution and post-processing scripts producing useful extra outputs. This also makes all tests work with our monitoring web pages (ART monitor) which will allow validation shifters to monitor the results of these tests. This point addresses ATR-19434.
- Add a protection disabling
--imf
if IMF is not available, to address ATR-19717. - Write a new script
runTrigART.py
to replace trigger developer workflow based on TUT unit test with an equally convenient alternative.
New instructions for developers
- No need to check out TrigUpgradeTest unless you are modifying something in this package.
- After compilation, still run ctest in the build directory, but this should only run real unit test for the components you're building. No large athena jobs.
- In a new directory run
runTrigART.py -m TrigUpgradeTest
to run a minimal set of TUT tests (only the two which run in CI) or runrunTrigART.py TrigUpgradeTest
(without -m) to run all ART tests from TrigUpgradeTest. SeerunTrigART.py -h
for help. The-j N
option for running multiple tests in parallel is available. - Read the summary of test results printed by the test and verify if everything was fine. You can also browse full logs, root files, etc. in the subdirectories created by the script.
The script functionality will be extended to other validation packages (e.g. TriggerTest) when we apply the new naming conventions there.
Example output:
$ runTrigART.py -j 2 -m TrigUpgradeTest
INFO The following 2 tests will be executed:
INFO test_trigUpgr_full_menu_build.sh
INFO test_trigUpgr_newJO_build.sh
INFO Executing ART command: art.py run -q --max-jobs=2 --type=build . results
Execute: /cvmfs/atlas-nightlies.cern.ch/repo/sw/master/2019-05-01T2135/Athena/22.0.2/InstallArea/x86_64-centos7-gcc8-opt/bin/./art-internal.py build job . results . out build 0Execute:
/cvmfs/atlas-nightlies.cern.ch/repo/sw/master/2019-05-01T2135/Athena/22.0.2/InstallArea/x86_64-centos7-gcc8-opt/bin/./art-internal.py build job . results . out build 1
Execute: mkdir -p results
INFO ART finished, analysing the results
INFO ----------------------------------------------------------------------------------------------------------
INFO | test_trigUpgr_newJO_build.sh | Configuration: 0, athena: 0, CheckLog: 0, RegTest: 0, RootComp: 999 |
INFO | test_trigUpgr_full_menu_build.sh | athena: 0, CheckLog: 0, RegTest: 0, RootComp: 999 |
INFO ----------------------------------------------------------------------------------------------------------
INFO RootComp failed in 2 tests, but this step is currently ignored
INFO ==================================================
INFO All 2 executed tests succeeded
INFO ==================================================