Skip to content

Unit, system and performance tests

Simon Spannagel requested to merge unittests into master

This MR extends our CI to include many tests of the framework, broadly categorized in the following types:

Module Tests

Tests of individual modules. Using a fixed seed (random_seed = 0) together with a fixed Geant4 version will always reproduce the same event. One event is produced and the DEBUG level output of the respective module is checked using regex.

The CMakeLists.txt automatically globs for all configurations found in the test_modules folder and executes Allpix Squared with them. The regex to search for are also stored in the configuration file as comment (starting line with # , no other comments in the file allowed) and will be picked up by CMake automatically.

Currently implemented tests:

  • test_01_geobuilder.conf
  • test_02-1_electricfield_linear.conf
  • test_02-2_electricfield_init.conf
  • test_03_deposition.conf
  • test_04-1_propagation_project.conf
  • test_04-2_propagation_generic.conf
  • test_05_transfer_simple.conf
  • test_06_digitization.conf
  • test_07_histogramming.conf
  • test_08-1_writer_root.conf
  • test_09_reader_root.conf

Dependencies between test can be indicated using the #DEPENDS keyword in the config file. See test 09 for an example, it uses the output of test 08-1 to read data back in.

Performance Tests

Similar to the implementation above, performance tests use configurations prepared such, that one particular module takes most of the load ("slowest instantiation"), and a couple of thousand events are simulated from a fixed seed. The #TIMEOUT keyword in the configuration file will ask CTest to abort the test after the given running time.

Since fluctuations on the CI runners occur, arising from different loads of the executing machines, all performance CI jobs are marked with the allow_failure: true keyword. This will allow GitLab to continue processing the pipeline but will mark the end-result as "passed with warnings" indicating an issue in the pipeline. This should then be checked manually.

Performance tests are now limited to "native" runners, i.e. they are not executed on docker hosts where the hypervisor decides on the number of parallel jobs. Currently this comprises only the Mac runners, but a native CentOS7 host should be up soon, tagged for performance tests.

Current tests comprise:

  • test_01_deposition.conf
  • test_02-1_propagation_generic.conf
  • test_02-2_propagation_project.conf
  • test_02-3_propagation_generic_multithread.conf

Core Tests

to be implemented

Edited by Simon Spannagel

Merge request reports