Skip to content
Snippets Groups Projects

Run all tests

Merged Teng Jian Khoo requested to merge khoo/easyjet:run-all-tests into master

Addresses #79 (closed) by adding an 'all' option to easyjet-test.

Running all the tests in serial was very slow, so I have parallelised the execution using xargs.

In some cases one might not want to wait for all tests to complete to see if any failed. For this purpose, the all-early-exit option quits the submission loop as soon as any test fails. Some child processes may continue to run after the script itself terminates.

Each test runs in its own directory and has a log file capturing the job output. Tests run and exit codes are summarised like so:

> easyjet-test -d ../run/test-all all-exit-early
running hh4b-mc21-syst-hist
getting file mc21_13p6TeV.601479.PhPy8EG_HH4b_cHHH01d0.DAOD_PHYS_10evts.e8472_s3873_r13829_p5631.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
running sh4b-boosted-mc20-fast-awkward
getting file mc20_13TeV.801636.Py8EG_A14NNPDF23LO_XHS_X3000_S70_4b.DAOD_PHYS_10evts.e8448_a899_r13145_p5631.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
running sh4b-data22
getting file data22_13p6TeV.00436656.physics_Main.DAOD_PHYS_10evts.r14190_p5632.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
running ttbar-mc21-fast
getting file mc21_13p6TeV.601237.PhPy8EG_A14_ttbar_hdamp258p75_allhad.DAOD_PHYS_10evts.e8453_s3873_r13829_p5631.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
running data22-fast
running lite-sh4b-boosted-mc20
getting file data22_13p6TeV.00436656.physics_Main.DAOD_PHYS_10evts.r14190_p5632.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
getting file mc20_13TeV.801636.Py8EG_A14NNPDF23LO_XHS_X3000_S70_4b.DAOD_PHYSLITE_10evts.e8448_a899_r13145_p5631.pool.root from https://gitlab.cern.ch/easyjet/hh4b-test-files/-/raw
xargs: bash: exited with status 255; aborting
sh4b-data22 completed successfully
hh4b-mc21-syst-hist completed successfully
data22-fast completed successfully
ttbar-mc21-fast completed successfully
sh4b-boosted-mc20-fast-awkward completed successfully
failed tests detected
Waiting 10s to collect test results from other incomplete jobs

Available test results:

data22-fast succeeded
hh4b-mc21-syst-hist succeeded
lite-sh4b-boosted-mc20 failed with exit code 1
sh4b-boosted-mc20-fast-awkward succeeded
sh4b-data22 succeeded
ttbar-mc21-fast succeeded

With all-exit-early, there is no guarantee that all job results are available at the time that the test summary is collected. However, with the option all, xargs waits for all results, and should therefore print a complete summary.

Edited by Teng Jian Khoo

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Dan Guest resolved all threads

    resolved all threads

  • Dan Guest marked this merge request as draft

    marked this merge request as draft

  • Dan Guest
  • Teng Jian Khoo added 4 commits

    added 4 commits

    • 58389a53 - 1 commit from branch easyjet:master
    • 17f43af0 - Add crudely parallelised all-tests option to easyjet-test
    • 8c7aff57 - Fix printouts
    • de7718d2 - Use xargs for parallelisation

    Compare with previous version

  • Teng Jian Khoo added 1 commit

    added 1 commit

    Compare with previous version

  • Teng Jian Khoo changed the description

    changed the description

  • Teng Jian Khoo added 1 commit

    added 1 commit

    Compare with previous version

  • Teng Jian Khoo marked this merge request as ready

    marked this merge request as ready

  • Dan Guest
  • Dan Guest
  • Dan Guest
  • Dan Guest
    • Resolved by Teng Jian Khoo

      You probably won't love this suggestion, but rather than exporting just about everything in this script, I wonder if it might be a bit easier to either:

      • Make it loop over the modes and call itself in each mode (should be argument $0 or ${BASH_SOURCE[0]} or something). The script itself is completely self contained so exports won't be needed.
      • Write another script for the "all" version. This could even call the "all script".
  • Dan Guest
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading