Move HltEffChecker scripts onto RDataFrame

This MR replaces the existing ROOT TTree interface of the HltEfficiencyChecker scripts (e.g. hlt_calculate_rates.py) with RDataFrame for speedup. This MR also contains a reworking of the fitting to allow the user to specify if plots should be produced per denominator or with all denominators shown together (thanks @nschulte for the idea and implementation!).

The performance numbers can be found posted below and performance checks are summarised in the checklist below:

• Performance comparable with master
• Efficiencies consistent with master
• Overlaps consistent with master
• Rates consistent with master

added RTA label

assigned to @jagoodin

marked this merge request as draft

added 3 commits

• def34472 - Additional tweaks to hlt_calculate_rates
• 8c3de940 - Replace TTree interface in efficiencies and overlap scripts
• 2a52508a - Preliminary replacement of TTree in plotting

Compare with previous version

added 1 commit

• af77a3db - Fixes to ensure RDF pointers dereferenced

Compare with previous version

added 1 commit

• ed2d78ff - Fixed formatting

Compare with previous version

added 2 commits

• 8be06d8c - Fixes to use of RDF pointers
• 3b527c56 - Reduced efficiency calculation calls into a single event loop

Compare with previous version

added 1 commit

• 8b370d7f - Reduce historgamming to two event loops in efficiencies script

Compare with previous version

added 2 commits

• 35aa443b - Reduce rates script to a single event loop call
• 38adda92 - Reduce overlap tool to two event loops per configuration

Compare with previous version

added 4 commits

• 95355d59 - Include fix to efficiencies calculation in plotting
• fdac3dbd - Tidying of overlap tool script
• cec2071a - Fixes to legends in plotting, configuration of event loops
• fe9a75c6 - Included changes to plotting from @nschulte, added argument to enable denom grouping

Compare with previous version

added 1 commit

• bde966c2 - Fixed formatting

Compare with previous version

added 2 commits

• 33861032 - Fixes to event number pointer dictionary use
• b41ad4e6 - Move pointers from lists to dicts in rates

Compare with previous version

added 1 commit

• 76d2b1fb - Fixed formatting

Compare with previous version

added 1 commit

• 5470e580 - Fixes to printing in overlap script

Compare with previous version

changed the description

changed the description

I've run the tests locally 5 times on the master branch and 5 times on the branch from this MR; this is how the new branch compares to master:

Hi @rjhunter, this MR is ready for review I've posted the performance with respect to master master on local tests in the thread above; looks fairly comparable to master on the smaller event numbers/line numbers of the tests, but would expect the parallel behaviour to kick in on larger samples.

added 2 commits

• 44afcae3 - Include interface to specify number of threads for RDataFrame
• 5267731b - Remove remaining print in efficiencies script

Compare with previous version