Add function moduleExists to test if a module exists without hiding
other errors.
Should be compatible with both py2 and 3.

Remove a couple debugging printouts.

 - Get basestring from past.builtins.
 - Iterator interface.
 - Fix raw byte string comparison.
 - Work around root issue with converting strings stored in trees.

scheduled to be written and not yet available in StoreGate.

John Kenneth Anders authored 5 years ago and Atlas Nightlybuild committed 5 years ago

Looking for patches using WorkDir_DIR instead of TestArea (ATLASJT-399)

See merge request atlas/athena!26078

(cherry picked from commit 1b47900188d5881d467ad756e9b069220d466e97)

8fb2f94d Looking for patches using WorkDir_DIR instead of TestArea (ATLASJT-399)

Various fixes for python3 compatibility.

John Derek Chapman authored 5 years ago and Walter Lampl committed 5 years ago

This commit adds an `m_doNoise` configurable property to the
`FrontEndSimTool` base class. This is used to determine whether
or not the `ThermalNoise(...)` and `RandomNoise(...)` functions
should be called in the derived classes. This should replicate
the behaviour in `21.0` where the `PixelNoisyCellGenerator` is
only used when `digitizationFlags.doInDetNoise()==True`.

Running Digi_tf.py leads to currentSubstep=None in athenaExecutor._prepAthenaCommandLine
There was already a workaround for this for two cases, now added protections for the
checks for --threads and --nprocs.

AODtoTAG step has been removed from the q-tests, so don't try to validate
it here.  Otherwise, we get spurious failures.

John Kenneth Anders authored 5 years ago and Atlas Nightlybuild committed 5 years ago

Added q440 test to TCT for pileup check

See merge request atlas/athena!25053

(cherry picked from commit ca23d022d2d9e6ab58542487c018514a6d6a88a3)

115da193 Added q440 test to TCT for pileup check

If you do a SetBinError with an out-of-bounds bin, older versions of root
would still create the error array in the histogram.  LWHists had code
to emulate this behavior.  However, this changed as of root 6.18: if the 
bin is out of bounds, then the histogram is not changed.  Adapt the LWHist
code to match.

This branch disables the AODtoTAG substep and replaces it with a
minimal dummy process that produces an empty output file for the
TAG data type. It was desired to be able to run jobs requiring a
TAG output as part of general standarized q-tests, but without the
AODtoTAG substep actually being executed. Given that the transform
will check for each output file passed as an argument, the only way to
disable the substep was to replace it with a dummy one, which
produces an empty file with a name set by the argument:
--outputTAGFile.
Producing no output of type TAG was not an option since file validation
cannot be turned off for specific output types

The previous commit by mistake removed the installation of the
`python/scripts` directory leading to failures of `acmd.py` (ATEAM-550).

As it was described in the How to run multi-core jobs in production
document, we are introducing two command-line options for the job transform:
	--multithreaded
	--multiprocess

As the names suggest, the former is used for running AthenaMT and the latter
for running AthenaMP. The number of threads/processes is taken from a new
ATHENA_CORE_NUMBER environment.

Now, we have two equivalent ways to run a job in AthenaMT
(Reco_tf is used here as an example):

ATHENA_CORE_NUMBER=N Reco_tf.py --multithreaded [Other Options]
Reco_tf.py --athenaopts=--threads=N [Other Options]

Similarly, for running AthenaMP we have two options:

ATHENA_CORE_NUMBER=N Reco_tf.py	--multiprocess [Other Options]
Reco_tf.py --athenaopts=--nprocs=N [Other Options]

We will get a configuration error from the transform if one of the following occurs:

* ATHENA_PROC_NUMBER was not set but either --multiprocess or --multithreaded was provided
* Both --multiprocess and --multithreaded options were provided to the same job
* --multiprocess was used in combination with either --athenaopts=--nprocs=N or --athenaopts=--threads=N
* --multithreaded was used in combination with --athenaopts=--threads=N or --athenaopts=--nprocs=N

By implementing	these changes we declare the old ATHENA_RPOC_NUMBER environment
obsolete. Athena repository has	been cleaned up	from the dependency on this
environment. The only exception is Generator code, which is yet to be migrated.

The plugins have been moved to External/flake8_atlas in
atlasexternals (atlas/atlasexternals!483).

Several python packages were used from the now removed AtlasPyFwdBwdPorts
package. As such they it didn't need an explicit dependency. All these
packages are now available via LCG and should be declared as explicit
dependency to ensure they get installed.