Implemented new interface to Job Transforms for activating MT and MP jobs
As it was described in the How to run multi-core jobs in production document and in ATLASJT-403, we are introducing two command-line options for the job transform:
--multithreaded
--multiprocess
As the names suggest, the former is used for running AthenaMT
and the latter for running AthenaMP
. The number of threads/processes is taken from a new ATHENA_CORE_NUMBER
environment variable.
Now, we have two equivalent ways to run a job in AthenaMT
(Reco_tf
is used here as an example):
ATHENA_CORE_NUMBER=N Reco_tf.py --multithreaded [Other Options]
Reco_tf.py --athenaopts=--threads=N [Other Options]
Similarly, for running AthenaMP
we have two options:
ATHENA_CORE_NUMBER=N Reco_tf.py --multiprocess [Other Options]
Reco_tf.py --athenaopts=--nprocs=N [Other Options]
We will get a configuration error from the transform if one of the following occurs:
-
ATHENA_CORE_NUMBER
is set but neither--multiprocess
nor--multithreaded
option was provided. -
ATHENA_CORE_NUMBER
was not set but either--multiprocess
or--multithreaded
was provided - Both
--multiprocess
and--multithreaded
options were provided to the same job -
--multiprocess
was used in combination with either--athenaopts=--nprocs=N
or--athenaopts=--threads=N
-
--multithreaded
was used in combination with--athenaopts=--threads=N
or--athenaopts=--nprocs=N
By implementing these changes we declare the old ATHENA_RPOC_NUMBER
environment
obsolete. Athena repository has been cleaned up from the dependency on this
environment. The only exception is Generator code, which is yet to be migrated.
Tag: @emoyse @wlampl @ssnyder @leggett @fwinkl @akraszna @girolamo @elmsheus @tmaeno @pnilsson @walker @mavogel (please feel free to forward this MR to other interested people).