Skip to content

Transform support for ComponentAccumulator based configs

Walter Lampl requested to merge wlampl/athena:transformWtihCA1 into master

First attempt to support ComponentAccumulator-based jobs via jobTransforms. Change as little as possible on the transform side, maintain backward compatibility. This MR contains elements from !27212 (closed).

In more detail:

  • Add a new optional argument --CA to both athena.py and the transform.
  • When athena.py is called with --CA it stops after steps dealing with $LD_PRELOAD and delegates execution to the ThinCAWrapper.sh script discussed in !27212 (closed).
  • When the transform is called with --CA, it writes a slightly different runArgs.py file: At the end of it imports a method called fromRunArgs from the skeleton-file and executes `fromRunArgs(RunArgs).

So we are not longer calling athena.py with two scripts that get executed in one global name-space but only one python script that calls the second one.

What's left to be done is to provide new skeleton files for the various processing steps. The new skeletons must implement a method fromRunArgs(RunArgs) that translates the run-args into ConfigFlags and then calls the necessary configuration methods to do the job it's supposed to do.

Currently, the skeleton file names contain dots (like skeleton.RAWtoESD_tf.py) which make python import awkward. The new skeletons files should not have dots in their name.

One could argue that the RunArgs is an unnecessary intermediate step. It is kept to minimize changes and help ensuring backward compatibility.

Edited by Walter Lampl

Merge request reports