Re-scale histograms if postprocessed n-tuples are incomplete
In the histogramming step, say one job failed out of 60. We have 59 jobs that succeeded and 1 that failed. If we know the number of raw NanoAOD events that went into the failed job from the start (at the start of the skimming stage), vs. the total number of raw NanoAOD events (at the start of the skimming stage), say the failed job-chain (skim_0.root
-> postprocessed_ntuple_0.root
) was responsible for 1/60th of the initial dataset, then we can take the successful jobs and scale them up by (60/59)
(i.e. 1/(59/60)
, or 1/(fraction of initial NanoAOD events that were successfully processed)
.
Need to make sure this doesn't clash with the postprocess step that computes this weighing.
Alternatively, find a way to resubmit just one postprocess job.