Skip to content

Major rewrite of TrigUpgradeTest - remove ctests

Rafal Bielski requested to merge rbielski/athena:remove-TUT-unit-tests into master

Apologies for the large MR, but this is a major disruption and needs to be done all in one go. Since many files are renamed and then modified, GitLab diff displays some of them as deleted and new files added.

Description

This MR addresses a long-standing issue of TrigUpgradeTest and the trigger developer workflow abusing the unit test functionality to run large integration tests running athena with tens of components. This is what ART is for. Thus, this MR properly interfaces all TrigUpgradeTest tests to the trigger validation infrastructure and at the same time removes them from the list of unit tests. When this is merged, the tests will be only ART tests. I also provide a new script for developers to run all or selected tests easily and see the summary of the results.

Changes

In more details, this MR includes the following changes:

  1. Remove nearly all tests from TUT unit tests in CMakeLists.txt. The remaining few are real unit tests, although they test units from other packages and should be moved in subsequent MRs.
  2. Rename all test scripts to implement new unified naming scheme proposed in ATR-19735.
  3. Interface all tests to trigger validation infrastructure including common execution and post-processing scripts producing useful extra outputs. This also makes all tests work with our monitoring web pages (ART monitor) which will allow validation shifters to monitor the results of these tests. This point addresses ATR-19434.
  4. Add a protection disabling --imf if IMF is not available, to address ATR-19717.
  5. Write a new script runTrigART.py to replace trigger developer workflow based on TUT unit test with an equally convenient alternative.

New instructions for developers

  1. No need to check out TrigUpgradeTest unless you are modifying something in this package.
  2. After compilation, still run ctest in the build directory, but this should only run real unit test for the components you're building. No large athena jobs.
  3. In a new directory run runTrigART.py -m TrigUpgradeTest to run a minimal set of TUT tests (only the two which run in CI) or run runTrigART.py TrigUpgradeTest (without -m) to run all ART tests from TrigUpgradeTest. See runTrigART.py -h for help. The -j N option for running multiple tests in parallel is available.
  4. Read the summary of test results printed by the test and verify if everything was fine. You can also browse full logs, root files, etc. in the subdirectories created by the script.

The script functionality will be extended to other validation packages (e.g. TriggerTest) when we apply the new naming conventions there.

Example output:

$ runTrigART.py -j 2 -m TrigUpgradeTest
INFO     The following 2 tests will be executed: 
INFO         test_trigUpgr_full_menu_build.sh
INFO         test_trigUpgr_newJO_build.sh
INFO     Executing ART command: art.py run -q --max-jobs=2 --type=build . results
Execute: /cvmfs/atlas-nightlies.cern.ch/repo/sw/master/2019-05-01T2135/Athena/22.0.2/InstallArea/x86_64-centos7-gcc8-opt/bin/./art-internal.py build job  . results . out build 0Execute: 
/cvmfs/atlas-nightlies.cern.ch/repo/sw/master/2019-05-01T2135/Athena/22.0.2/InstallArea/x86_64-centos7-gcc8-opt/bin/./art-internal.py build job  . results . out build 1
Execute: mkdir -p results
INFO     ART finished, analysing the results

INFO     ----------------------------------------------------------------------------------------------------------
INFO     | test_trigUpgr_newJO_build.sh     | Configuration: 0, athena: 0, CheckLog: 0, RegTest: 0, RootComp: 999 |
INFO     | test_trigUpgr_full_menu_build.sh | athena: 0, CheckLog: 0, RegTest: 0, RootComp: 999                   |
INFO     ----------------------------------------------------------------------------------------------------------
INFO     RootComp failed in 2 tests, but this step is currently ignored

INFO     ==================================================
INFO     All 2 executed tests succeeded
INFO     ==================================================

Merge request reports