Changes to handling of nEventsPerJob
Changes
- Removal of
-toption - Modification of check
CountHepMC == nEventsPerJobtoCountHepMC == Requested output events(which should always be satisfied, hence the removal of the-toption) -
nEventsPerJobFromJOis introduced and always read from the jO file - The extrapolation of the CPU time now happens automatically if
nEventsPerJobFromJO != nEventsPerJobFrom(where the latter is read fromlog.generate)
Tests
Using 700078 prepared by @fsiegert:
- Using
-maxEvents=100andNEventsPerJob=1000, committing with the commit script
./scripts/commit_new_dsid.sh -d=700078 -n
Generate params:
---------------------
- nEventsPerJob = 1000
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- actual CPU (100 events) = 9.71 hrs
- CPU extrapolated to 1000 events = 97.1 hrs
- CPU = 97.10 hrs <-- ERROR: Too high CPU time - should be between 6-12h. Adjust nEventsPerJob!
- estimated CPU for CI job = 0.97 hrs
-
Standalone
logParserrun without the jO present in the directory
logParserinternally assignsnEventsPerJobFromJO=nEventsPerJob(wherenEventsPerJobis read from theevgenConfig.nEventsPerJobprinted inlog.generate).
python scripts/logParser.py -i 700xxx/700078/log.generate
---------------------
Generate params:
---------------------
- nEventsPerJob = 1000
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- actual CPU (100 events) = 9.71 hrs
- CPU extrapolated to 1000 events = 97.1 hrs
- CPU = 97.10 hrs <-- ERROR: Too high CPU time - should be between 6-12h. Adjust nEventsPerJob!
- estimated CPU for CI job = 0.97 hrs
-
Standalone
logParserrun pointing to a jO file in a different directory
Here I put the jO file in999999and usedevgenConfig.nEventsPerJob = 100. The jO in700078still hasevgenConfig.nEventsPerJob = 1000:
python scripts/logParser.py -i 700xxx/700078/log.generate -j 999xxx/999999/mc.Sh_228_yyy_01NLO.py
---------------------
Generate params:
---------------------
- nEventsPerJob = 100
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- CPU = 9.71 hrs
- estimated CPU for CI job = 0.10 hrs
- Virtual memory = 3061.180 Mb
Todo
-
Update README
Closes #73 (closed), #81 (closed)
Edited by Spyros Argyropoulos