Skip to content

sweep PileupReweightingTool from 21.2

Nils Erik Krumnack requested to merge krumnack/athena:prw-cherry-2020-06-23 into master

Note to Merge Review Shifters: This is a cherry-pick/sweep from 21.2, not actually new code. Please only make the minimal set of comments necessary, do not perform a full/regular review.

I hope I managed to get the actual cherry-pick correct, the package had been copied over from SVN to both 21.2 and master independently (and not always mentioning the version in SVN either), and there were a number of commits that touched a large number of files in the repository. There was one memory management update/bug-fix I copied over manually afterwards, hopefully there isn't anything else I missed.

List of commits cherry-picked:

  • e9a28a7a Upgraded to PileupReweighting-00-04-04.
  • d9026f9f changed C to cxx
  • 94e0e860 Update PileupReweightingTool.cxx - Remove the default config files, since they aren't appropriate for MC16 (release 21) analysis, so were just going to lead to mistakes by being there.
  • dd757f11 added warning for people who continue to use default config files ... they need to be weened off these
  • a979fa9c fixed a bug in the configuration of the systematic variation subtools in standard (multiperiod) setup
  • 5dd8af11 new property of tool for VaryRandomRunNumber
  • 9ee2d14b added two new diagnostic scripts.
  • fd63fdfa removed obsolete default config files
  • 174a0dca default to MC16 config if not providing any config files but you are providing lumicalc files
  • 17f9af04 Update generatePRW.py - stop the SystemExit exception being caught so that the error messages aren't printed even when the script succeeds
  • c0a5eb27 Update debugExample.C - make simple example work in release 21.2
  • 61a0ff2b now will check for existing NTUP_PILEUP (which are complete) and will advise user to download these instead of submitting unnecessary jobs
  • fd1fbf38 changed name of output stream from METADATA to NTUP_PILEUP, to avoid confusion .. some users thought there could be a difference when in fact they are the same output
  • 7e3f637c only display warning if user has actually loaded a default config file, at the moment it will display the warning even if they haven't but they are about to have an exception thrown at them
  • f29156fe add ut test as an executable, since the input file it needs is not available to others
  • 89842cd7 remove premature exit from initialize method ... this is causing incorrect configuration in config-fileless mode
  • 9c36234d added test for data weights
  • 562a94eb add test for run-dependent data weights
  • b641c512 make weights obtained when prescaling MC be 'run-dependent' (if doing mu-dependent). This might be preferred method in the future
  • 99b0444e make new run-dependent prescale weights an option, not the default. Since I realised how to work things properly with the old way of doing things
  • a61d18a5 change filenames of files generated in each test, in attempt to avoid tests failing (possibly clobbering one another)
  • 32560d1e change argument names in methods, as requested by shifters
  • 3b288521 improve error message
  • 9349afb1 improvements to generate and check PRW scripts ... main change for end users is generatePRW will no longer ignore partially available datasets, because it seems that a decent number of our official NTUP_PILEUP are only partially available at the moment
  • 1d610e01 RECOMMENDATION CHANGE: scale factor moved to 1.03 +/- 0.04
  • 0e1356fe fix the test by specifying the scale factor in the test, since the result depends on it
  • bb1e5253 Add PURW actual mu generator
  • 463de309 Add PURW actual mu generator
  • a776f1de Respond to code review
  • f09be107 added support for recaling to actual mu, which will use the special config files being prepared by data prep
  • be082634 attempt to fix problem reported by Tadej Novak when unprescaling data that has been redirected by unrepresenteddataaction=3
  • ff064157 use unique ptr and only fill actual mu for none trigger
  • 0b51f1a6 remove commented code
  • 61f84398 handle fact that actual mu config files are not normalized in the same way as lumicalc files. This was leading to overrun issues in random run number generation
  • d85c5ef7 catch uses of incorrect mu profile (MC16c config files with MC16d files), instead of ERROR and continue, we now throw exceptions
  • bfad2b7a catch case of giving non-lumicalc files to LumiCalcFiles property
  • 4402c91f fixed tests by removing calls that will now throw an exception
  • b0c0deb6 added an exception catch test
  • d0319263 Update TPileupReweighting.cxx - remove line at request of reviewer
  • 3ef16bba Revert "Merge branch 'Merge21.0Into21.2-20180523' into '21.2'"
  • 9a8faeb5 Update debugExample.C - remove what was bad c++ invoking the constructor of AAH
  • 11636c25 Revert "Merge branch 'revert-b8857cf6' into '21.2'"
  • e636e514 improved messaging around checks for unrepresented data, to better indicate the usual cause of the problem
  • 3da4841e made period assignments part of period configuration a configurable property of the tools, and added the MC16e runNumber (310000) as part of that assignment, assigning it to 2018 run numbers
  • c3ca5430 Update generatePRW_jobOptions.py - add the error handler, should be using this in the production jobs for safety.
  • d6807b1b Update generatePRW_jobOptions.py - had to remove the default period assignments for config file generating.
  • 8be72582 Update generatePRW.py - fix issue with datasets that end with r tag (new format)
  • c8b062f3 Add argument to checkPRW.py for keeping suspect channels in output file
  • 6da8a6da fix case where user is reusing the unprescaled lumicalc files for unprescaled triggers in mc/data prescaling/unprescaling. Turns out the standard unprescaled lumicalc files in group area have prescale settings equivalent to disabled triggers :-S
  • 1f0f1031 Update TPileupReweighting.cxx - put back in comment line
  • be7a9740 now that period configs are put in parent tool by default, we need to catch the bad periods a bit more verbosely
  • c545059d convenenient output file with rucio commands is produced
  • 5b093c60 update normalisation cut-off
  • 9c16e7a9 Adding some trivial updates to remove GCC 8 warnings.
  • d80c5e10 Watching for PRW that has already been applied
  • a57d11e7 remove deprecated pathena option
  • 8f8df786 Adding PRW auto-configuration
  • dd72e6cb Patch for PRW auto-configuration running on data
  • f118f6c1 allow manual input
  • 8804568d only set None values, not all if one is None
  • 95f8865f Fix a typo in line 18 and a logic problem in line 57 - ATLASG-1526
  • 09bdfa5d Ident line18 in the if statement
  • 1116e892 There is a special case that needs some additional protection If you have the sum of weights ==0 for a given mu then you will have 0 in the PRW profile leading to an exception or FPE later
  • 6346cab5 Update PRWtool unit test as a exception is now no longer thrown in this situation
  • 31f93c1a Update TPileupReweighting.cxx so that it does not enforce a minimum normalization, but rather checks that a histogram does not have an integral of exactly zero.

Merge request reports