Transform support for ComponentAccumulator based configs
First attempt to support ComponentAccumulator-based jobs via jobTransforms. Change as little as possible on the transform side, maintain backward compatibility. This MR contains elements from !27212 (closed).
In more detail:
- Add a new optional argument
--CA
to both athena.py and the transform. - When athena.py is called with
--CA
it stops after steps dealing with$LD_PRELOAD
and delegates execution to theThinCAWrapper.sh
script discussed in !27212 (closed). - When the transform is called with --CA, it writes a slightly different runArgs.py file: At the end of it imports a method called
fromRunArgs
from the skeleton-file and executes `fromRunArgs(RunArgs).
So we are not longer calling athena.py with two scripts that get executed in one global name-space but only one python script that calls the second one.
What's left to be done is to provide new skeleton files for the various processing steps. The new skeletons must implement a method fromRunArgs(RunArgs)
that translates the run-args into ConfigFlags and then calls the necessary configuration methods to do the job it's supposed to do.
Currently, the skeleton file names contain dots (like skeleton.RAWtoESD_tf.py
) which make python import awkward. The new skeletons files should not have dots in their name.
One could argue that the RunArgs is an unnecessary intermediate step. It is kept to minimize changes and help ensuring backward compatibility.