Fix warnings about regexp syntax.

Frank Winklmeier authored 4 years ago and Walter Lampl committed 4 years ago

Delete unused fileutils, path and merge_join modules. Also delete pep8
and avn alias. The pep8 script is superseded by flake8. And the avn
alias was pointing to a non-existent script.

Work around pyroot issue with getting trailing garbage when accessing
a string stored in a tree.   This can confuse the bytes->unicode conversion
resulting in a crash.

 - it.next() -> next(it)

With some python3 versions, importing past emits deprecation warnings.

As it was described in the How to run multi-core jobs in production
document, we are introducing two command-line options for the job transform:
	--multithreaded
	--multiprocess

As the names suggest, the former is used for running AthenaMT and the latter
for running AthenaMP. The number of threads/processes is taken from a new
ATHENA_CORE_NUMBER environment.

Now, we have two equivalent ways to run a job in AthenaMT
(Reco_tf is used here as an example):

ATHENA_CORE_NUMBER=N Reco_tf.py --multithreaded [Other Options]
Reco_tf.py --athenaopts=--threads=N [Other Options]

Similarly, for running AthenaMP we have two options:

ATHENA_CORE_NUMBER=N Reco_tf.py	--multiprocess [Other Options]
Reco_tf.py --athenaopts=--nprocs=N [Other Options]

We will get a configuration error from the transform if one of the following occurs:

* ATHENA_PROC_NUMBER was not set but either --multiprocess or --multithreaded was provided
* Both --multiprocess and --multithreaded options were provided to the same job
* --multiprocess was used in combination with either --athenaopts=--nprocs=N or --athenaopts=--threads=N
* --multithreaded was used in combination with --athenaopts=--threads=N or --athenaopts=--nprocs=N

By implementing	these changes we declare the old ATHENA_RPOC_NUMBER environment
obsolete. Athena repository has	been cleaned up	from the dependency on this
environment. The only exception is Generator code, which is yet to be migrated.

Tamara Vazquez Schroeder authored 7 years ago and Atlas Nightlybuild committed 7 years ago

Transform support for input files specified as WebDAV TURLs (ATLASJT-376)

See merge request atlas/athena!8074

(cherry picked from commit 1495609e [formerly 18565a5281bf9877b7d84e11de667c7df56b7638])

6ff72308 Transform support for input files specified as WebDAV TURLs (ATLASJT-376)
9c792014 Transform support for input files specified as WebDAV TURLs (ATLASJT-376)
05cc1a4d Clean up of commented code

Former-commit-id: 0f15d55f

Former-commit-id: 2db867087177e7a27331043fccb840ecacfd65d2

	* python/AthFileLite.py: Port back changes from PyUtils-00-14-90 to support https and rucio redirector
	* branch tag PyUtils-00-14-85-04


Former-commit-id: fdc42192

2016-04-27 Walter Lampl <walter.lampl@ cern.ch>
	* rewrite of get-tag-diff.py to be cmt and cmake compatible
	* tag PyUtils-00-14-75

2016-04-12  scott snyder  <snyder@bnl.gov>

	* Tagging PyUtils-00-14-74.
	* python/RootUtils.py: Remove diagnostics.  Keep temp files open
	through the end of program execution to avoid cling caching
	issues.

2016-04-11  scott snyder  <snyder@bnl.gov>

	* Tagging PyUtils-00-14-73.
	* Still working on ATN test failure.  Seems to be related to
	ROOT-7059.

2016-04-11 Will Buttinger <will@cern.ch>
	* bin/getAMIDatasetParameters.py: improved default printout, preserve order of specified datasets, and preserve comments/blanklines from file in the inDsTxt
	* Tagging PyUtils-00-14-72
...
(Long ChangeLog diff - truncated)

	* merged diff between PyUtils-00-14-37 PyUtils-00-14-38:
	* python\PoolFile.py (extract_streams_from_tag): workaround for
	PyROOT getattr(tree, branchname) bug in ROOT 5.34.25 for reading
	Tokens from TAG files (https://its.cern.ch/jira/browse/ATLASRECTS-1489)
	* Tagging PyUtils-00-14-34-01

2015-06-04  Marcin Nowak  <Marcin.Nowak@cern.ch>
	* bin\getTagDiff.py: improved handling of LCG and MIG tags
	* Tagging PyUtils-00-14-34

2015-06-01  Marcin Nowak  <Marcin.Nowak@cern.ch>
	* python\scripts\diff_root_files.py: fix --ignore-leaves which
	broke in the last tag
	* Tagging PyUtils-00-14-32
	* make --ignore-leaves able to ignore subbranches
	* Tagging PyUtils-00-14-33

2015-05-28  Marcin Nowak  <Marcin.Nowak@cern.ch>
	* python\scripts\diff_root_files.py: rewrote the code to
	handle differencies better
...
(Long ChangeLog diff - truncated)

reporting, throw proper exceptions.
Prevent AthFileLits from running AthenaMP mode (PyUtils-00-14-07)

	* python/AmiLib.py
	- Remove commented out code
	* Tagging as PyUtils-00-14-07

2014-12-18 Graeme Stewart <graeme.andrew.stewart@cern.ch>
	* python/AmiLib.py
	- Another big refactoring:
	- badresult improved and now returns a boolean and an error message
	- Rename dry_run to dryrun for consistency
	- Introduce PyUtilsAMIException exception, which now gets thrown
	  consistently if there are problems
	- Consolidate the functions of get_project_of_pkg, get_version_of_pkg
	  and tc_submit_tag._get_projects into single method get_pkg_info
	- Rewrite get_version_of_pkg_with_deps to use module utilities
	- Simplify find_pkg to remove callback function
	* python/scripts/tc_find_pkg.py
	* python/scripts/tc_find_tag.py
	- Embed script in try: except PyUtilsAMIException:
	* python/scripts/tc_show_clients.py
	- Update to get_pkg_info (N.B. Script is still non-functional and disabled)
...
(Long ChangeLog diff - truncated)