Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • L LHCb
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 193
    • Issues 193
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
  • Jira
    • Jira
  • Merge requests 44
    • Merge requests 44
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • LHCbLHCb
  • LHCb
  • Merge requests
  • !3116

Draft: Migrate PyConf to GaudiConfig2

  • Review changes

  • Download
  • Email patches
  • Plain diff
Closed Alex Pearce requested to merge apearce-pyconf-gaudiconfig2 into master Jun 25, 2021
  • Overview 23
  • Commits 5
  • Pipelines 4
  • Changes 18

This allows for the gains discussed in #141. It's also noticeable faster to execute the GaudiConfig2-based configuration than the Configurables-based one! 🚀

Some specific changes that were necessary:

  • Migrate the DDDBConf and CondDB user configurables over to a GaudiConfig2-based helper method in PyConf.database.
  • Migrate IOHelper to GaudiConfig2-based helper method. and PyConf.io.
  • Introduce a PyConf.Services importer which wraps 'service configurables'.
    • The importer doesn't actually wrap the import objects; I'm not convinced it's necessary to have this importer but it does make it less confusing rather than having to import configurables from two different places (PyConf.X and GaudiConfig2.Configurables).

As a result PyConf.components.setup_component has been removed. I also addressed #127 (closed) as I was touching PyConf a lot anyway.

Obviously this looks like a big change but the goal is to produce exactly the same configuration, modulo some names and locations. I've checked this for the example.py file with an without input, but I would be more comfortable checking against a larger application, i.e. Moore. Of course that means a lot more work, but we need to transition Moore before merging this anyway.

To do

  • Fix pickle deserialisation for diff test.
    • Requires gaudi/Gaudi#196 (closed).
  • Decide on how to migrate 'traditional' options files (those not import'able).
    • As a stop-gap, to prevent us needing to convert every Moore options file, we could introduce a wrapper which accepts a GaudiConfig2-based configuration dict and instantiates old Configurable objects based on that.
    • gaudi/Gaudi#192 (closed) might resolve this.
  • Decide on a clean merge_config strategy for certain properties (particularly ApplicationMgr.ExtSvc).
  • AFAIK the DD4Hep configuration is not tested, as it's behind a 'feature flag' which no test enables.
  • Handle IOVLock in dataflow_config.apply.
  • Migrate from underscore-based imports (Gaudi__SomeAlg) to package based ones (Gaudi.SomeAlg).
    • GaudiConfig2.Configurables supports the latter, but I added support in the PyConf importers for the former so that I wouldn't have to change all the imported names (being lazy).
Edited Jul 22, 2021 by Alex Pearce
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: apearce-pyconf-gaudiconfig2