Skip to content
Snippets Groups Projects
Commit d483859c authored by Frank Winklmeier's avatar Frank Winklmeier
Browse files

CITest: update documentation

Update the README and the text that is displayed at the start of each
CI test.
parent cef99ca4
No related branches found
No related tags found
4 merge requests!69091Fix correlated smearing bug in JER in JetUncertainties in 22.0,!58791DataQualityConfigurations: Modify L1Calo config for web display,!51674Fixing hotSpotInHIST for Run3 HIST,!50909CITest: update documentation
......@@ -3,7 +3,19 @@ these as they have a significant impact on the CI turnaround time.
[TOC]
## Test definition
# Running the tests
After compiling this package locally, individual tests can be run specifying the test name (supports regex):
```sh
ctest -R CITest_RecoRun2Data
```
For builds involving several packages, tests can be selected/excluded using the "CITest" label:
```sh
ctest -L CITest # run all CI tests
ctest -LE CITest # run all tests, except CI tests
```
# Adding new tests
- Test are defined in separate files for each project (e.g. [`Athena.cmake`](Athena.cmake)).
- Tests should have a short self-explanatory name. Do not add the word "test" to the name itself.
- If tests depend on each other consider using a common basename and delimit the steps with an underscore,
......@@ -51,12 +63,12 @@ Rather than matching the number of actual cores used, this number should reflect
expected system load. E.g. if a job runs with 8 threads but the system load during running is
significant lower, one can reduce this number to allow other jobs to run in parallel.
### Post-processing
## Post-processing
All tests defined with `atlas_add_citest` run a [default post-processing script](cmake/citest_post.sh.in)
that checks the log file for errors using [`noerror.sh`](../TestTools/).
An additional post-processing script can be specified with the
`POST_EXEC_SCRIPT` keyword. The overall test result will be the exit code of that
post-processing script! The orignal test result is stored in the
post-processing script! The original test result is stored in the
`${ATLAS_CTEST_TESTSTATUS}` environment variable and can be used in the post-processing
script if needed.
......@@ -70,28 +82,8 @@ New post-processing scripts should be made as general as possible and named as
[`test/checkXYZ.sh`](test/).
## Running the tests
The tests in this package can be run locally like any other unit tests after compiling
the package locally and running `ctest`.
All tests in this package carry the "CITest" label and therefore they can be included/excluded
in `ctest` runs, via:
```sh
ctest -L CITest # only run CI tests
ctest -LE CITest # run all tests, except CI tests
```
To avoid running the CI tests as part of the regular unit testing in the nightly build,
the tests are disabled by default. To enable them (e.g. for CI builds), the following
needs to be added to the `cmake` command line:
```sh
cmake -DATLAS_ENABLE_CI_TESTS=TRUE ...
```
This is done already by default for partial `WorkDir` builds so a regular developer does
not need to worry about this.
## Internals
# Internals
- All tests are run in a separate working directory in the build area: `AtlasTest/CITest/CMakeFiles/ciTestRun/<test>/` where `<test>` is the name used in `atlas_add_citest`.
- The main test log file within the working directory is named `<test>.log`.
- If you want to verify the final command that is being run (useful e.g. to debug issues with quotes), check the content of `AtlasTest/CITest/test-bin/<test>.exe` in the build area.
- To avoid running the CI tests as part of the regular unit testing in the nightly build, the tests are disabled by default. To enable them (e.g. for CI builds), one has to configure with `cmake -DATLAS_ENABLE_CI_TESTS=TRUE ...`. This is done by default for partial `WorkDir` builds so a regular developer does not need to worry about this.
......@@ -34,7 +34,6 @@ if [ ${status} -ne 0 ]; then
EOF
fi
# This time stamp is used by NICOS:
echo "[CITest][${ATLAS_CTEST_TESTNAME}] end time: `date` (`date +%s.%N`)"
......
......@@ -5,8 +5,18 @@
# Pre-exec script used for all CI tests
#
branch=${gitlabTargetBranch:-${AtlasBuildBranch}}
if [ -n "${branch}" ]; then
url="https://gitlab.cern.ch/atlas/athena/-/tree/${branch}/AtlasTest/CITest"
else
url="the AtlasTest/CITest package"
fi
# This time stamp is used by NICOS:
echo "[CITest][${ATLAS_CTEST_TESTNAME}] start time: `date` (`date +%s.%N`)"
echo "[CITest][${ATLAS_CTEST_TESTNAME}] on machine `hostname`"
echo "[CITest][${ATLAS_CTEST_TESTNAME}] to run this test locally see ${url}"
echo "----------------------------------------------------------"
# Clean private working directory:
if [ -n "@CI_WORKDIR@" ]; then
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment