Skip to content

Print memory and RDataFrame graph statistics

Pieter David requested to merge piedavid/bamboo:memprofile into master

From investigating #46 (closed) (I also did a bit of refactoring of the backend, to have the RDataFrame methods being called from fewer places - that makes it easier to collect statistics and try things). The suggestion from the ROOT developers to use the Histo{1,2}D call that doesn't need the JIT is also there (it moves some memory allocation to an earlier stage, and saves a bit of CPU time later on).

INFO-level output now looks like this:

INFO:bamboo.analysismodules:168 plots defined in 82.90s, max RSS: 3770.72MB
INFO:bamboo.analysismodules:Number of uses per node type: defaultdict(<class 'int'>, {'gInterpreter_Declare': 45, 'Filter': 339, 'Define': 1997, 'Histo1D': 2886, 'Histo2D': 222})
INFO:bamboo.analysismodules:HistoND calls per column type: defaultdict(<class 'int'>, {('ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter,void>', ('double',), 'float'): 888, ('ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter,void>', ('float',), 'float'): 1998, ('ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter,void>', ('float', 'float'), 'float'): 222})
INFO:bamboo.analysismodules:Systematic shape variations impacting any plots: pileup, muid, muiso, mumutrig, btagging2016, jer, jesTotal, unclustEn, btagging, elid, eleltrig, elmutrig, mueltrig
INFO:bamboo.analysismodules:Starting to fill plots
INFO:bamboo.analysismodules:Plots finished in 588.89s, max RSS: 8018.63MB (3108 histograms)

I also included a few unrelated small fixes (typos etc.)

Edited by Pieter David

Merge request reports