Expanded Validation for lhcb-dpa/project#93

Related to #32, lbapcommon#8 (closed), lbapcommon#11 (closed).

I'm wondering if it might be worth it that for each validation we do we add a corresponding option in the info.yaml to allow users to configure exactly how it works. Having discussed with @cburr we will try to do most of this using the checks framework This means we will run most of it after the jobs have been tested so that validation issues that don't break jobs will be picked up but still allow the DaVinci options etc... to be tested as well to try have a single CI give as much information about issues with the production as possible. Some bits of validation like checking the yaml is valid yaml etc... will still happen beforehand.

For example:

  • Check duplicate bk_query: Strict/Lenient/None
  • Check polarity matches between job and input: Strict/Lenient/None
  • Check all requested trees are present: Strict/Lenient/None
  • % of events in MCDecayTree: Float in [0,100] and Strict/Lenient/None

where Strict/Lenient/None means:

  • Strict: Fail the test
  • Lenient: Print warnings
  • None: Don't do it

To do:

Edited by Aidan Richard Wiederhold