razumov created page: CTests authored by Ivan Razumov's avatar Ivan Razumov
CTest for a generator is defined in [lcgcmake/generators/CMakeLists.txt](generators/CMakeLists.txt) beside the definition of the generator itself.
If an executable is to be compiled for the <generator>, it is done by
`lcgcmake/generators/<generator>/tests/CMakeLists.txt`. The test code
(if it is not taken from the installed generator's `share/sources`)
is also put in this directory.
The `CMakeLists.txt` for the test compilation uses the `CMake` system
`FindPackage`. Some macros to find well known packages are defined
in the standard `CMake` (see (http://www.cmake.org/cmake/help/v2.8.9/cmake.html)[CMake documentation]). If the tested package
depends on some other package of `GENSER` or external, additional
macros `FindPackage` are used, e.g. `FindHepMC`, `FindThePEG`. They can
be defined in 2 places:
1. package `cmaketools` defined in `lcgcmake` and installed in `/external`
2. `lcgcmake/cmake/modules`. In this case this additional search path is
added inside `lcgcmake/generators/<generator>/tests/CMakeLists.txt`
Usually new `Find*` macros are put in the second location, and after
full testing added to `cmaketools` (kept in git).
When everything is prepared, a single ctest can be run in lcgcmake-build
by
ctest -R <generator>
It is possible to increase verbosity of the test like this:
ctest -VV -R <generator>
CMakeLists for the test compilation can be debugged using messages:
`message(STATUS "my message text ${LIBLIST}")`
It is recommended to avoid scripts in the ctests system
Once the test is defined in `lcgcmake/generators/CMakeLists`, it will be run in
the nightlies.
Tests can have labels: LABELS PhysicsCheck. This is not used now.
It is possible to specify maximal time: TIMEOUT 3000
Default time unit is seconds.
### CTESTS NAMING CONVENTION:
Each test name contains
* generator's name
* generator's version
* type of test
* "native" tests are test that use examples, sources, ... provided by package developers
* "genser" tests are test that use source codes made by GENSER team
* some tests may have a short name accompanying to its function, mainly these tests check common functionality of Installation or provide specific actions.
* definition of the test. It should be related with test's actions and/or parameters.
Formula of test name: `generator-version.(native|genser)-definition[-rivet for rivet tests]`
Physics rivet tests convention:
1. Reference histograms are kept in `lcgcmake/generators/reference`
2. At the beginning of each reference yoda file there is a line like below, with a keyword "Reference_info":
# Reference_info : obtained 09.03.2015 with alpgen 2.1.4 and agile 1.4.1 on x86_64-slc6-gcc48-opt, experimental
# Process Z + 2jets, electrons (steering file inputalpgen), warm 3x100000, 5000000,
# hadronized and read by rivet 10000 events
Tools available for the histograms comparison:
1. `${CMAKE_SOURCE_DIR}/cmake/scripts/yoda2root.cmake -DAIDA2ROOT=${AIDA2ROOT}`, example is
......
......