Skip to content
Snippets Groups Projects
Commit 740ec516 authored by James Catmore's avatar James Catmore
Browse files

Change keyword for setting output formats in Derivation_tf

Currently the keyword for setting the output DAOD formats in the new CA-based derivation framework transform `Derivation_tf.py` is `requiredDerivedFormats`. We have agreed that this is too long and since the keyword `formats` is available, we've decided to switch to this. Consequently once this MR is merged the new transform would be used as follows:

```
Derivation_tf.py --CA --inputAODFile aod.pool.root --outputDAODFile daod.pool.root --formats PHYS PHYSLITE ETC
```

Since this new transform isn't used in production yet, this MR will have no impact on any existing production or activities.
parent 9cf56627
No related merge requests found
......@@ -40,8 +40,8 @@ def fromRunArgs(runArgs):
ConfigFlags.Input.Files = getattr(runArgs, f'input{allowedInputTypes[idx]}File')
# Output formats
if hasattr(runArgs, 'requiredDerivedFormats'):
logDerivation.info('Will attempt to make the following derived formats: {0}'.format(runArgs.requiredDerivedFormats))
if hasattr(runArgs, 'formats'):
logDerivation.info('Will attempt to make the following derived formats: {0}'.format(runArgs.formats))
else:
logDerivation.error('Derivation job started, but with no output formats specified - aborting')
raise ValueError('No derived formats specified')
......@@ -70,8 +70,8 @@ def fromRunArgs(runArgs):
cfg.merge(PoolReadCfg(ConfigFlags))
cfg.merge(PoolWriteCfg(ConfigFlags))
if hasattr(runArgs, 'requiredDerivedFormats'):
for formatName in runArgs.requiredDerivedFormats:
if hasattr(runArgs, 'formats'):
for formatName in runArgs.formats:
derivationConfig = getattr(DerivationConfigList, f'{formatName}Cfg')
cfg.merge(derivationConfig(ConfigFlags))
else:
......
......@@ -9,7 +9,7 @@ from PyJobTransforms.trfExe import reductionFrameworkExecutor
def addDerivationArguments(parser):
"""Add common derivation command-line parser arguments."""
parser.defineArgGroup('Derivation', 'Derivation Options')
parser.add_argument('--requiredDerivedFormats', nargs='*',
parser.add_argument('--formats', nargs='*',
type=argFactory(argList),
help='List of required D(2)AOD formats',
group='Derivation')
......
......@@ -18,7 +18,7 @@ The examples are as follows:
In run 3 the derivation framework will move to the component accumulator and will be build differently. This package also contains examples showing how to run this new component accumulator based derivation framework. Unlike the examples above the config files are found in the python/ directory and are all python modules rather than job options. Usage is:
`Derivation_tf.py --CA --inputAODFile aod.pool.root --outputDAODFile test.pool.root --requiredDerivedFormats TEST1 TEST3 ...`
`Derivation_tf.py --CA --inputAODFile aod.pool.root --outputDAODFile test.pool.root --formats TEST1 TEST3 ...`
The examples are very similar to the ones above:
......
......@@ -1788,8 +1788,8 @@ class reductionFrameworkExecutor(athenaExecutor):
self.setPreExeStart()
msg.debug('Preparing for execution of {0} with inputs {1} and outputs {2}'.format(self.name, input, output))
if 'NTUP_PILEUP' not in output:
# New derivation framework transform uses "requiredDerivedFormats"
if 'reductionConf' not in self.conf.argdict and 'requiredDerivedFormats' not in self.conf.argdict:
# New derivation framework transform uses "formats"
if 'reductionConf' not in self.conf.argdict and 'formats' not in self.conf.argdict:
raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_REDUCTION_CONFIG_ERROR'),
'No reduction configuration specified')
......@@ -1799,7 +1799,7 @@ class reductionFrameworkExecutor(athenaExecutor):
formatList = []
if 'reductionConf' in self.conf.argdict: formatList = self.conf.argdict['reductionConf'].value
if 'requiredDerivedFormats' in self.conf.argdict: formatList = self.conf.argdict['requiredDerivedFormats'].value
if 'formats' in self.conf.argdict: formatList = self.conf.argdict['formats'].value
for reduction in formatList:
if ('DAOD' in output):
dataType = 'DAOD_' + reduction
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment