Skip to content

Draft: Migrate PyConf to GaudiConfig2

Alex Pearce requested to merge apearce-pyconf-gaudiconfig2 into master

This allows for the gains discussed in #141. It's also noticeable faster to execute the GaudiConfig2-based configuration than the Configurables-based one! 🚀

Some specific changes that were necessary:

  • Migrate the DDDBConf and CondDB user configurables over to a GaudiConfig2-based helper method in PyConf.database.
  • Migrate IOHelper to GaudiConfig2-based helper method. and PyConf.io.
  • Introduce a PyConf.Services importer which wraps 'service configurables'.
    • The importer doesn't actually wrap the import objects; I'm not convinced it's necessary to have this importer but it does make it less confusing rather than having to import configurables from two different places (PyConf.X and GaudiConfig2.Configurables).

As a result PyConf.components.setup_component has been removed. I also addressed #127 (closed) as I was touching PyConf a lot anyway.

Obviously this looks like a big change but the goal is to produce exactly the same configuration, modulo some names and locations. I've checked this for the example.py file with an without input, but I would be more comfortable checking against a larger application, i.e. Moore. Of course that means a lot more work, but we need to transition Moore before merging this anyway.

To do

  • Fix pickle deserialisation for diff test.
  • Decide on how to migrate 'traditional' options files (those not import'able).
    • As a stop-gap, to prevent us needing to convert every Moore options file, we could introduce a wrapper which accepts a GaudiConfig2-based configuration dict and instantiates old Configurable objects based on that.
    • gaudi/Gaudi#192 (closed) might resolve this.
  • Decide on a clean merge_config strategy for certain properties (particularly ApplicationMgr.ExtSvc).
  • AFAIK the DD4Hep configuration is not tested, as it's behind a 'feature flag' which no test enables.
  • Handle IOVLock in dataflow_config.apply.
  • Migrate from underscore-based imports (Gaudi__SomeAlg) to package based ones (Gaudi.SomeAlg).
    • GaudiConfig2.Configurables supports the latter, but I added support in the PyConf importers for the former so that I wouldn't have to change all the imported names (being lazy).
Edited by Alex Pearce

Merge request reports