Add ATLAS specific tasklist handers

Adds two tasklist handlers for running ATLAS workflows:

• TransformTaskList for running transforms
• AthenaTaskList for running athena.

Input

Both tasklist handlers assume that the tasklist is a list of input files to run over.

Automatic splitting an input file into multiple tasks is supported via the maxEventsPerJob setting. This option specifies the number of events per task. The handler counts the number of events in the input file and splits based on the requested number of events per task. This requires PyROOT as a dependency, but not the entire ATLAS software.

However, when possible, it is recommended to use AthenaMP for parallelizing event processing. This has a reduced memory footprint.

Output

The output is created in the current working directory. On successful exit, the output is then copied to the workdir using rsync. This two stage process is required due to how AthenaMP determines its temporary outputs. The implication is that the runner needs to run the command using bash.

Configuring Athena

All arguments to the transform or athena are defined as extra keys in the tasklist hander definitions. For example, adding postInclude = HITtoRDO:Campaigns/MC16NoPileUp.py translates into an argument of --postInclude='HITtoRDO:Campaigns/MC16NoPileUp.py'. Note the automatic wrapping of the value string inside single quotes. These are automatically added by the tasklist handler.

There are a few protected or keywords: path, workdir and maxEventsPerJob. For TransformTaskList, the following are also special: transform, input and output.

See examples below for the meaning of protected keywords. Note that path and workdir are protected due to PyTaskFarmer's internal mechanics.

TransformTaskList Example

Special required keywords:

• transform: Name of transform to run (ie: Reco_tf.py)
• input: Type of input files (ie: HITS)
• output: Type of output files (ie: RDO)

A simple example for running no pileup digitization is below:

[digi]
TaskList = taskfarmer.atlas.TransformTaskList
transform = Reco_tf.py
athenaopt = all:--nprocs=64
input = HITS
output = RDO
autoConfiguration = everything
digiSteeringConf = StandardInTimeOnlyTruth
conditionsTag = default:OFLCOND-MC16-SDR-RUN2-06
geometryVersion = default:ATLAS-R2-2016-01-00-01
postInclude = default:PyJobTransforms/UseFrontier.py
preInclude = HITtoRDO:Campaigns/MC16NoPileUp.py
preExec = all:from ParticleBuilderOptions.AODFlags import AODFlags; AODFlags.ThinGeantTruth.set_Value_and_Lock(False);' 'HITtoRDO:from Digitization.DigitizationFlags import digitizationFlags; digitizationFlags.OldBeamSpotZSize = 42

AthenaTaskList Example

A simple example for running IDPVM:

[valid]
TaskList = taskfarmer.atlas.AthenaTaskList
jobOptions = InDetPhysValMonitoring/InDetPhysValMonitoring_topOptions.py
output = M_output.root
command = from InDetPhysValMonitoring.InDetPhysValJobProperties import InDetPhysValFlags; InDetPhysValFlags.doValidateTruthToRecoNtuple.set_Value_and_Lock(True)