Skip to content

Spring cleaning & Enforce tests for all configs

Nikita Ivvan Pondrequested to merge
npond/training-dataset-dumper:executable_cleanup into main

Description

In continuation of my quest to cleanup the dumper, I have now also implemented the dump-jer executable and jetm2-jer config to work with dump-single-btag. I then noticed that we have a config jetm2-flow which didn't have a test associated with it. I wasn't sure what executable or test file this should be run with, which is why this MR is starting in draft.

As a result of my frustration, I decided to try and implement a test which ensures that

  • All config files have at least 1 test in the test-dumper associated with it
  • All tests in the test-dumper appear in the .gitlab-ci file, ensuring all tests are being run.

This does mean we also need to add tests for a series of configs which don't currently have tests.

In total, this merge request changes the following:

  • Removes some old batch condor code which was no longer in use
  • Removed the analysisbase setup script - everything should now either be athanalysis or athena
  • Delete some executables (dump-jer, dump-multi-config, dump-upgrade-hi) in favour of dump-single-btag : ca blocks were added to the respective configs
  • Remove dump-trigger-workingpoints and its associated config, as no longer in use.
  • Consolidtae dump-trigger-emtopo, dump-trigger-pflow, dump-trigger-tau into the single previous executable dump-trigger-all
  • Add 2 new CA blocks, the MCTruthClassifier and PoorMansIpAugmenter to allow for some of the above-described executable cleanup
  • Removed a bunch of old configs that either are no longer used, or which are covered by other configs (e.g. removal of EMPFlow_softlep or the flows config, as both are included in GN3_dev
  • Added new test which ensures that all configs in the main config directory must have a test in the test-dumper associated.
  • Added tests for any configs which didn't have them previously.
  • Update grid-submit to use correct updated executables

A noteworthy bug was found, some weird precision issue which seems to come just from updating a trigger test file. This is documented in #168

Review checklist:

  • CI Passing
  • Comments addressed
  • Source branch is up to date with target
Edited by Nikita Ivvan Pond

Merge request reports