Putting into CI and review now, as agreed at software coordination on 17th March. Note this does touch code that is used for all DAOD production (specifically the SlimmingHelper=, but should have zero impact on any existing DAODs.
This is the first of several MRs to migrate the derivation framework to the component accumulator. This targets three aspects:
- Slimming mechanism
- Job transforms mechanism
- Updated DerivationFrameworkExamples
The SlimmingHelper is retained but now has two modes of operation - with and without the component accumulator.
- In the case where the component accumulator is used, the SlimmingHelper is supplied the list of container names and types (needed to build the ItemList) in the input file via the ConfigFlag
Input.TypedCollections. The ItemList is then returned by the SlimmingHelper so that it can be included in the output stream via the new CA-compliant
OutputStreamConfig.pyhere. This has been modified to allow the setting of an AcceptAlg for filtering
- In the second case run in the old style, the list is generated inside the SlimmingHelper from the non-CA method
inputFileSummary['eventdata_items'], as before. The ItemList is directly added to the stream.
- The existing job options still work as the method names are retained. The constructor now accepts an extra keyword argument
NamesAndTypes, and has a new method to return the ItemList called GetItemList.
- At the same time, the machinery for blocking certain containers from being added as
AllVariablesis removed, as it has no relevance in Run 3.
Basic job transforms mechanism is included:
- transform is
Derivation_tf.pywith the format(s) specified with
requiredDerivedFormats(equivalent to the old
- follows example of the OverlaySkeleton with a DerivationSkeleton
- this skeleton works out the input/output filenames and decides which workflow to use: AOD->DAOD, EVNT->DAOD or DAOD->D2AOD
- loops over the formats required by
requiredDerivedFormatsand adds the config for each format to the component accumulator
- the configs for each format are made into python modules in
pythoninstead of job options in
Derivation_tf.py --CA --inputAODFile AOD.27089396._001365.pool.root.1 --outputDAODFile test.pool.root --requiredDerivedFormats TEST1 TEST3 --maxEvents 1000
The examples are being progressively updated to use the new CA set up. The naming convention will reflect the one used in the rest of the framework, that is the config python file will have the same name as the format itself (e.g. TEST1.py, TEST3.py, etc)