Skip to content

Speed up python configuration

Rosen Matev requested to merge rmatev-python-optimizations into 2017-patches

A collection of changes that speeds up the whole python configuration by more than a factor of six. Now HLT1 alone takes a few seconds and HLT2 less than 40 s.

There are two categories. Neither modifies functional behaviour. First, very safe changes that are quite foolproof:

  • Clean up imports: from Configurables import X is super costly if called in the innermost loop
  • Avoid unnecessary getProp call in __allConfiguredLocations: @apearce not sure why this was there, it shouldn't make a difference
  • Cache PV3D bindMembers instance: Unsurprisingly, this is called often and hence is costly
  • Use the smarter recursion everywhere for overriding props: The recursion is much faster as it only visits each node once.
  • Refactor bindMembers._InputOutputLocationMatchMaker: The core is unchanged, but now is more readable and paranoid.

And the slightly less fool proof changes:

  • _Cache result of OutputLocationsGetter: In theory this can miss problems if the application of configurables is really messed up. There are tests for the latter so I think this is reasonably safe. OTOH, it only saves 7 out of 42 s.
  • Reduce the deepcopying in allCuts: Slightly less aggressive deepcopy. One can imagine a line author doing something very funky that will screw things up (change a nested common cut within a stage). @raaij you might want too look at this one.

The refactoring of _InputOutputLocationMatchMaker and some extra paranoia triggered !425 (merged), hence this MR depends on and contains the fix in !425 (merged)

profile.svg

Edited by Rosen Matev

Merge request reports