Add check for ATHENA_PROC_NUMBER

Changes introduced

• Check if the jO tries to modify the environment variable ATHENA_PROC_NUMBER
• Harmonise calls to jo_utils functions for easier maintenance

Tests

1. Set ATHENA_PROC_NUMBER using os.environ then unset it => Should fail

JO content:

os.environ["ATHENA_PROC_NUMBER"] = "64"

# Go to serial mode for Pythia8
if 'ATHENA_PROC_NUMBER' in os.environ:
    print 'Noticed that you have run with an athena MP-like whole-node setup.  Will re-configure now to make sure that the remainder of the job runs serially.'
    njobs = os.environ.pop('ATHENA_PROC_NUMBER')
    # Try to modify the opts underfoot
    if not hasattr(opts,'nprocs'): print 'Did not see option!'
    else: opts.nprocs = 0
    print opts

Output:

python scripts/check_jo_consistency_main.py 421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py

===> Checking jobOption consistency...

Modified jO file: 421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py - jO file = mc.MGPy8EG_A14NNPDF23_tWgamma_art.py
        ...
	ERROR: file /Users/sargyrop/Desktop/mcjoboptions/scripts/../421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py tries to set ATHENA_PROC_NUMBER to 64. This is not allowed!

2. Set ATHENA_PROC_NUMBER using environ instead of os.environ => should fail

from os import environ
environ["ATHENA_PROC_NUMBER"] = "42"

gives

ERROR: file /Users/sargyrop/Desktop/mcjoboptions/scripts/../421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py tries to modify ATHENA_PROC_NUMBER from None to 42. This is not allowed!

3. Set ATHENA_PROC_NUMBER using os.environ => Should fail

Moving os.environ["ATHENA_PROC_NUMBER"] = "64" to the end of the jO:

python scripts/check_jo_consistency_main.py 421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py

===> Checking jobOption consistency...


Modified jO file: 421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py - jO file = mc.MGPy8EG_A14NNPDF23_tWgamma_art.py
	...
	ERROR: file /Users/sargyrop/Desktop/mcjoboptions/scripts/../421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py tries to modify ATHENA_PROC_NUMBER from None to 64. This is not allowed!

4. Preset ATHENA_PROC_NUMBER that is deleted in jO => Should succeed

Succeeds because the script does not care about preset environment variables.

5. Setting ATHENA_PROC_NUMBER arbitrarily named environment => should fail

from os import environ as env1
env1["ATHENA_PROC_NUMBER"] = "64"

gives

ERROR: file /Users/sargyrop/Desktop/mcjoboptions/scripts/../421xxx/421006/mc.MGPy8EG_A14NNPDF23_tWgamma_art.py tries to modify ATHENA_PROC_NUMBER from None to 64. This is not allowed!

6. Using other ways to set ATHENA_PROC_NUMBER => should fail

var = "ATHENA_PROC_NUMBER"
env1[var] = "42"

and

env1.update({"ATHENA_PROC_NUMBER": "42"})

both fail

Caveats

The check does not cover:

• Preset environment variables which are changed on-the-fly when the jO is executed (it essentially starts from a clean environment) - see case 4 above
• if-else cases are not handled correctly: the script will assume that all lines of the jO will be executed. This will be very non-trivial to catch outside athena, so essentially this test must be replicated inside the transform if we want to be sure to catch these problems in the correct context

Issues resolved

Closes #125 (closed)