Extraction of nEventsPerJob from file

Bug

nEventsPerJob=5000 -> this is what the logParser reads from the file
# I can change it here
if (efficiencyLow) nEventsPerJob*=2
evgenConfig.nEventsPerJob = nEventsPerJob -> this is what the transform will use

Solution

Credits to Frank Sauerburger

Put the following in scriptB.py

import argparse

def readParamFromJO(jOpath, param):
    locals = {"evgenConfig": argparse.Namespace()}
    with open(jOpath) as jOFile:
        for line in jOFile.readlines():
            if "os.system" in line: continue # for security
            try:
                exec(line, {}, locals)
            except:
                # print(f"fail to parse {line}") # uncomment for debugging
                pass
    return getattr(locals["evgenConfig"], param) if hasattr(locals["evgenConfig"], param) else None

jOFile="./source/mc.scriptA.py"
nEventsPerJob=readParamFromJO(jOFile, 'nEventsPerJob')

# Check nEventsPerJob
if nEventsPerJob is None:
    print(f"WARNING: evgenConfig.nEventsPerJob is not defined in the jO. Will set to default=10000")
    nEventsPerJob=10000
else:
    print(f"nEventsPerJob from jO={nEventsPerJob}")

# Check minEvents
if readParamFromJO(jOFile, 'minEvents') is not None:
    print(f"ERROR: {jOFile} is using deprecated parameter evgenConfig.minEvents. Please switch to evgenConfig.nEventsPerJob")

Testing:

Put the following in ./source/mc.scriptA.py

import Sherpa_i.Sherpa_iConf
import os
import GeneratorFilters.GeneratorFiltersConf

include("./scriptA.py") # this doesn't work because python doesn't know what include is

evgenConfig.XVAR=5

filtSeq.YVAR=10

evgenConfig.nEventsPerJob=1
evgenConfig.nEventsPerJob=2
evgenConfig.nEventsPerJob*=3
evgenConfig.nEventsPerJob=os.system("rm test")
#evgenConfig.nEventsPerJob=10
#print(f"{evgenConfig.nEventsPerJob}")

Running python3 scriptB.py gives

fail to parse import Sherpa_i.Sherpa_iConf

fail to parse import GeneratorFilters.GeneratorFiltersConf

fail to parse include("./scriptA.py") # this doesn't work because python doesn't know what include is

fail to parse filtSeq.YVAR=10

Final Answer: nEventsPerJob=6

The added bonus is that if there is no evgenConfig.nEventsPerJob defined this would automatically throw an error.

What is done in ProdSys

The first occurence of evgenConfig.nEventsPerJob is used