Skip to content

Speed up PyConf with a cache of configurable calls

Rosen Matev requested to merge rm-pyconf-speedup into master

The added tests describe the behaviour. To spell out a few notable points:

  • Since bound parameters (with .bind) represent global state that can modify the behaviour of (nested) @configurable calls, they participate in the key of the cache (in addition to the function arguments).
  • If the arguments passed to @configurable functions cannot be serialized, caching is skipped and an INFO message is printed.
  • If the return valule of a @configurable function cannot be serialized, caching is skipped and an INFO message is printed. This is needed to make sure that cached return values are not mutated between calls.
  • If a return value is mutated between calls, an exception is raised. The user code needs to be fixed not to do that.

Configurable functions are not cached by default to make sure that we don't accidentally break things due to the introduction of side effects.

Miscellaneous improvements related to the speed of the configuration:

  • Better default python logging setup
  • Remove obsolete ROOT workarounds (and replace with another workaround for batch mode)
  • Avoid calls to getDefaultProperties for a small speedup
  • Limit profile on stdout to >10% cumtime
  • Update instructions to produce profile graph

Goes together with Moore!812 (merged), MooreAnalysis!35 (merged)

Edited by Rosen Matev

Merge request reports