Changes to handling of nEventsPerJob
Changes
- Removal of
-t
option - Modification of check
CountHepMC == nEventsPerJob
toCountHepMC == Requested output events
(which should always be satisfied, hence the removal of the-t
option) -
nEventsPerJobFromJO
is introduced and always read from the jO file - The extrapolation of the CPU time now happens automatically if
nEventsPerJobFromJO != nEventsPerJobFrom
(where the latter is read fromlog.generate
)
Tests
Using 700078
prepared by @fsiegert:
- Using
-maxEvents=100
andNEventsPerJob=1000
, committing with the commit script
./scripts/commit_new_dsid.sh -d=700078 -n
Generate params:
---------------------
- nEventsPerJob = 1000
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- actual CPU (100 events) = 9.71 hrs
- CPU extrapolated to 1000 events = 97.1 hrs
- CPU = 97.10 hrs <-- ERROR: Too high CPU time - should be between 6-12h. Adjust nEventsPerJob!
- estimated CPU for CI job = 0.97 hrs
-
Standalone
logParser
run without the jO present in the directory
logParser
internally assignsnEventsPerJobFromJO=nEventsPerJob
(wherenEventsPerJob
is read from theevgenConfig.nEventsPerJob
printed inlog.generate
).
python scripts/logParser.py -i 700xxx/700078/log.generate
---------------------
Generate params:
---------------------
- nEventsPerJob = 1000
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- actual CPU (100 events) = 9.71 hrs
- CPU extrapolated to 1000 events = 97.1 hrs
- CPU = 97.10 hrs <-- ERROR: Too high CPU time - should be between 6-12h. Adjust nEventsPerJob!
- estimated CPU for CI job = 0.97 hrs
-
Standalone
logParser
run pointing to a jO file in a different directory
Here I put the jO file in999999
and usedevgenConfig.nEventsPerJob = 100
. The jO in700078
still hasevgenConfig.nEventsPerJob = 1000
:
python scripts/logParser.py -i 700xxx/700078/log.generate -j 999xxx/999999/mc.Sh_228_yyy_01NLO.py
---------------------
Generate params:
---------------------
- nEventsPerJob = 100
- Requested output events = 100
- transform = Gen_tf
- inputFilesPerJob = 0
...
---------------------
Performance metrics:
---------------------
- CPU = 9.71 hrs
- estimated CPU for CI job = 0.10 hrs
- Virtual memory = 3061.180 Mb
Todo
-
Update README
Closes #73 (closed), #81 (closed)
Edited by Spyros Argyropoulos