Support executor substeps in transforms
For Run 3 we will need to smartly simulate and digitise jobs with 3 different beamspot sizes and pile-up profiles. This MR proposed an "executor substep" support for transforms, where the usual executor is split based on config.
A simple configuration function that returns number of steps and event fractions per step is needed, e.g.
def BeamspotSplitMC21a():
"""MC21a beamspot splitting configuration"""
substeps = 3
event_fractions = [0.3, 0.3, 0.4]
return substeps, event_fractionsEverything else is handled by individual transforms/skeletons. This configuration fragment is passed to the transform like --splitConfig 'HITtoRDO:Campaigns.BeamspotSplitMC21a'.
If number of substeps is larger than one (the trivial case is "useless"), then both executor steps (the execution graph) and executors (the actual steering of athena) are cloned and suffixed with ExecutorStep{i}. Each executor also knows its step index and the total number of steps. This and the config is also exposed to the Athena job in runArgs.
Each step runs as usual (but merging is never done, even with MP). After final steps all outputs (serial or MP) are merged only once.
Pile-up profile has an additional step entry that is used to filter. Sequential event numbering has also been enhanced to take into account total job size and local skip events.