Skip to content

Speed up expression and RDF graph construction

Pieter David requested to merge piedavid/bamboo:perfHH0608 into master

Performance improvements based on the HH use case (thanks @fbury ):

  • keep a 'visited' list when scanning for systematics nodes and range iterators
  • ... (maybe more, to be studied)

Especially for systematics on "deep" expressions (e.g. multivariate classifier outputs), where the full expression was still scanned, this can make a significant difference, e.g.

INFO:bamboo.analysismodules:58 plots defined in 2430.34s, max RSS: 5411.92MB
INFO:bamboo.dataframebackend:Number of uses per node type: 10120 Define, 83 gInterpreter_Declare, 3325 Filter, 8518 Histo1D, 988 Histo2D

becomes

INFO:bamboo.analysismodules:58 plots defined in 861.33s, max RSS: 5411.16MB
INFO:bamboo.dataframebackend:Number of uses per node type: 10120 Define, 83 gInterpreter_Declare, 3325 Filter, 8518 Histo1D, 988 Histo2D

for HH (1 category, 2 mass points). If more memory is used it should be short-lived, mostly.

Edited by Pieter David

Merge request reports