Extraction of nEventsPerJob from file
Bug
nEventsPerJob=5000 -> this is what the logParser reads from the file
# I can change it here
if (efficiencyLow) nEventsPerJob*=2
evgenConfig.nEventsPerJob = nEventsPerJob -> this is what the transform will use
Solution
Credits to Frank Sauerburger
Put the following in scriptB.py
import argparse
def readParamFromJO(jOpath, param):
locals = {"evgenConfig": argparse.Namespace()}
with open(jOpath) as jOFile:
for line in jOFile.readlines():
if "os.system" in line: continue # for security
try:
exec(line, {}, locals)
except:
# print(f"fail to parse {line}") # uncomment for debugging
pass
return getattr(locals["evgenConfig"], param) if hasattr(locals["evgenConfig"], param) else None
jOFile="./source/mc.scriptA.py"
nEventsPerJob=readParamFromJO(jOFile, 'nEventsPerJob')
# Check nEventsPerJob
if nEventsPerJob is None:
print(f"WARNING: evgenConfig.nEventsPerJob is not defined in the jO. Will set to default=10000")
nEventsPerJob=10000
else:
print(f"nEventsPerJob from jO={nEventsPerJob}")
# Check minEvents
if readParamFromJO(jOFile, 'minEvents') is not None:
print(f"ERROR: {jOFile} is using deprecated parameter evgenConfig.minEvents. Please switch to evgenConfig.nEventsPerJob")
Testing:
Put the following in ./source/mc.scriptA.py
import Sherpa_i.Sherpa_iConf
import os
import GeneratorFilters.GeneratorFiltersConf
include("./scriptA.py") # this doesn't work because python doesn't know what include is
evgenConfig.XVAR=5
filtSeq.YVAR=10
evgenConfig.nEventsPerJob=1
evgenConfig.nEventsPerJob=2
evgenConfig.nEventsPerJob*=3
evgenConfig.nEventsPerJob=os.system("rm test")
#evgenConfig.nEventsPerJob=10
#print(f"{evgenConfig.nEventsPerJob}")
Running python3 scriptB.py
gives
fail to parse import Sherpa_i.Sherpa_iConf
fail to parse import GeneratorFilters.GeneratorFiltersConf
fail to parse include("./scriptA.py") # this doesn't work because python doesn't know what include is
fail to parse filtSeq.YVAR=10
Final Answer: nEventsPerJob=6
The added bonus is that if there is no evgenConfig.nEventsPerJob
defined this would automatically throw an error.
What is done in ProdSys
The first occurence of evgenConfig.nEventsPerJob
is used