Expanded Validation for lhcb-dpa/project#93
Related to #32, lbapcommon#8 (closed), lbapcommon#11.
I'm wondering if it might be worth it that for each validation we do we add a corresponding option in the info.yaml
to allow users to configure exactly how it works. Having discussed with @cburr we will try to do most of this using the checks
framework This means we will run most of it after the jobs have been tested so that validation issues that don't break jobs will be picked up but still allow the DaVinci options etc... to be tested as well to try have a single CI give as much information about issues with the production as possible. Some bits of validation like checking the yaml is valid yaml etc... will still happen beforehand.
For example:
- Check duplicate bk_query: Strict/Lenient/None
- Check polarity matches between job and input: Strict/Lenient/None
- Check all requested trees are present: Strict/Lenient/None
- % of events in MCDecayTree: Float in [0,100] and Strict/Lenient/None
where Strict/Lenient/None means:
- Strict: Fail the test
- Lenient: Print warnings
- None: Don't do it
To do:
-
#32 -
lbapcommon#8 (closed) -
lbapcommon#11 -
lbapcommon#18 -
Add an optional validation of % of events in MCDecayTree -
Add the corresponding info.yaml
flags -
Add validation results to CI tests @roneil might need to help here