Skip to content
Snippets Groups Projects

TriggerMenuMT EventBuildingSeq: Migrating to CA and newJO (ATR-27050)

Merged Claire Antel requested to merge cantel/athena:23.0_tmmt_eventbuild_CA into 23.0

Purpose

  • Migrates EventBuildingSequence to ComponentAccumulator and make run for newJO menu.
  • Fixes config clash for oldJO (see ATR-27050?focusedComment)
  • Removes NoCAmigration exceptions (and catch thereof).

HLT configuration diffs

All diffs are related to name changing, in particular the additional suffix (_noSeed or _RoIBasedF) to distinguish between the different event builds, e.g.

                >>  only in 1st file :  ['PEBInfoWriterAlg/PEBInfoWriterAlg_PhysicsTLA']
                >>  only in 2nd file :   ['PEBInfoWriterAlg/PEBInfoWriterAlg_PhysicsTLA_noSeed']
...
                >>  only in 1st file :  ['PEBInfoWriterAlg/PEBInfoWriterAlg_JetPEBPhysicsTLA']
                >>  only in 2nd file :   ['PEBInfoWriterAlg/PEBInfoWriterAlg_JetPEBPhysicsTLA_RoIBasedFS']

also, renaming Sequence to mainSeq: pebSequence_AFPPEB -> pebMainSeq_AFPPEB

Affects

  • No effects: Should be same HLT configs, except for config clash fix.
  • Fixes configuration clash for old style affecting chains HLT_g3_etcut_LArPEB_L1EM3 and HLT_j25_LArPEBHLT_L1J15, as now InputMaker, RoIs and HypoAlg are uniquely named.
Edited by Claire Antel

Merge request reports

Pipeline #5700732 passed

Pipeline passed for d751e110 on cantel:23.0_tmmt_eventbuild_CA

Merged by Andrea CoccaroAndrea Coccaro 1 year ago (Jun 9, 2023 9:33am UTC)

Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Claire Antel changed target branch from master to 23.0

    changed target branch from master to 23.0

  • Author Developer

    Hi @fwinkl , When you are available couldyou have a look? This works most of the time, except for the Cosmics menu (e.g. TriggerMenuMT_HLT_Cosmic_run3_v1_ctest) which has issues configuring:

      File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/CommonSequences/EventBuildingSequences.py", line 32, in addEventBuildingSequence
        seq=menuSequenceCAToGlobalWrapper(pebMenuSequenceCfg, flags, chain=chain, eventBuildType=eventBuildType, chainDict=chainDict)
      File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/Config/MenuComponents.py", line 1103, in menuSequenceCAToGlobalWrapper
        return appendMenuSequenceCAToAthena(msca, flags)
      File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/Config/MenuComponents.py", line 1113, in appendMenuSequenceCAToAthena
        maker = conf2toConfigurable(msca.maker.Alg)
      File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-03-26T2101/Athena/23.0.22/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/LegacySupport.py", line 348, in conf2toConfigurable
        _areSettingsSame( existingConfigurable, comp, indent,  servicesOfThisCA )
      File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-03-26T2101/Athena/23.0.22/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/LegacySupport.py", line 278, in _areSettingsSame
        _areSettingsSame( existingVal, pvalue, indent,servicesOfThisCA)
      File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-03-26T2101/Athena/23.0.22/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/LegacySupport.py", line 178, in _areSettingsSame
        raise ConfigurationError("Old/new ({} | {}) cpp types are not the same for ({} | {}) !".format(
    
    AthenaConfiguration.ComponentAccumulator.ConfigurationError: Old/new (ViewCreatorPreviousROITool | ViewCreatorCentredOnIParticleROITool) cpp types are not the same for (ViewCreatorPreviousROITool/ViewCreatorPreviousROITool | ViewCreatorCentredOnIParticleROITool/ViewCreatorCentredOnIParticleROITool) !

    I thought maybe it's because the old code was only adding the input maker for a new sequence if the configured didn't exist already: https://gitlab.cern.ch/atlas/athena/-/blob/release/23.0.21/Trigger/TriggerCommon/TriggerMenuMT/python/HLT/CommonSequences/EventBuildingSequences.py#L260-267 while I add it always in def pebSequenceCfg(eventBuildType, inputMaker), because I thought it might be handled automatically. I am just not sure if I understand the old code as this seems like a genuine clash to me.

  • We have a situation here where:

    • The pebInputMaker and the pebInfoWriterTool are always named only according to the eventBuildType -- so this variable should encapsulate all configuration.
    • The pebInfoWriterTool is called to determine some of the options in pebInputMaker, but some other information is extracted based on the chain seeding.
    • Specifically, here the 'FSNOSEED' seeding is checked and used with _isRoIBasedPEB to determine if we should instead use the ViewCreatorCentredOnIParticleRoITool, without affecting any names.

    So we should resolve the clash if the _isFullScan and _isRoIBasedPEB block also triggered adding a suffix to the InputMaker name. It might be useful to find out exactly which config encounters this clash, in case there are some existing PEB streams that were wrong until now...

    Edited by Teng Jian Khoo
  • Author Developer

    Thanks, @khoo . I hope when I tried a hasty suffix change I was doing something silly since it was still clashing. I will do that, and track down the chain.

  • Claire Antel added 527 commits

    added 527 commits

    • e8c504c5...06ef8204 - 525 commits from branch atlas:23.0
    • 8a6a1967 - TMMT EventBuildingSeq: Migrating to CA and newJO, removing NoCAmigration...
    • 46daf19c - EventBuildingSequences: Avoiding cfg clashes by uniquely naming pebInputMaker and PEBInfoWriterAlg

    Compare with previous version

  • Claire Antel changed title from Draft: TriggerMenuMT EventBuildingSeq: Migrating to CA and newJO to Draft: TriggerMenuMT EventBuildingSeq: Migrating to CA and newJO (ATR-27050)

    changed title from Draft: TriggerMenuMT EventBuildingSeq: Migrating to CA and newJO to Draft: TriggerMenuMT EventBuildingSeq: Migrating to CA and newJO (ATR-27050)

  • Claire Antel added 1 commit

    added 1 commit

    • 3c30ee71 - EventBuildingSequences: uniquely naming RoIs

    Compare with previous version

    • Author Developer
      Resolved by Frank Winklmeier

      Hi, have now ensured that pebInputMaker and PEBInfoWriterAlg have unique names. But all is not well. While that has fixed the cosmics menu (though I still need to track down the cosmic chain that was causing the config clash so we know), it broke the TriggerMenuMT_menu_CA_PhysicsP1_pp_lowMu_run3_v1_ctest configuration test.

      The ctest the complaint is related to printing a property of a c++ tool (is this unit test new?):

      Traceback (most recent call last):
        File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/bin/test_menu_CA.py", line 40, in <module>
          menu = generateMenuMT(flags)
        File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/Config/GenerateMenuMT_newJO.py", line 105, in generateMenuMT
          menuAcc=makeHLTTree(flags)
        File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/Config/GenerateMenuMT_newJO.py", line 118, in makeHLTTree
          finalDecisions, menuAcc = decisionTreeFromChains(flags, steps, HLTMenuConfig.configsList(), HLTMenuConfig.dictsList(), newJO=False)
        File "/afs/cern.ch/work/c/cantel/private/AtlasEnv/AthenaDev/AthenaMenu/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/HLT/Config/ControlFlow/HLTCFConfig.py", line 390, in decisionTreeFromChains
          acc.printConfig(withDetails = True, summariseProps = True)
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 234, in printConfig
          printSeqAndAlgs(s, onlyComponents = onlyComponents)
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 222, in printSeqAndAlgs
          printSeqAndAlgs(c, nestLevel, onlyComponents = onlyComponents )
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 222, in printSeqAndAlgs
          printSeqAndAlgs(c, nestLevel, onlyComponents = onlyComponents )
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 222, in printSeqAndAlgs
          printSeqAndAlgs(c, nestLevel, onlyComponents = onlyComponents )
        [Previous line repeated 2 more times]
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 229, in printSeqAndAlgs
          printProperties(msg, c, nestLevel, printDefaults, printComponentsOnly)
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 61, in printProperties
          printProperties(msg, el, nestLevel+6)
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 60, in printProperties
          msg.info( "%s    * %s/%s", " "*(nestLevel+3), el.__cpp_type__, el.getName())
        File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-04-10T2101/Athena/23.0.24/InstallArea/x86_64-centos7-gcc11-opt/python/AthenaCommon/Configurable.py", line 352, in __getattr__
          return super( Configurable, self ).__getattribute__(attr)
      AttributeError: 'RegSelTool' object has no attribute '__cpp_type__'

      Am I missing a proper conversion?

  • Claire Antel added 1 commit

    added 1 commit

    • 7284e1f1 - GenerateMenuMT: Readd catch of TypeError exception with print to log and rethrow of exception

    Compare with previous version

  • Claire Antel resolved all threads

    resolved all threads

  • Claire Antel added 991 commits

    added 991 commits

    • 7284e1f1...e8155393 - 985 commits from branch atlas:23.0
    • 3dc480b4 - TMMT EventBuildingSeq: Migrating to CA and newJO, removing NoCAmigration...
    • 72464d68 - EventBuildingSequences: Avoiding cfg clashes by uniquely naming pebInputMaker and PEBInfoWriterAlg
    • bf44f18c - EventBuildingSequences: uniquely naming RoIs
    • 67a20bde - GenerateMenuMT: Readd catch of TypeError exception with print to log and rethrow of exception
    • 02c00355 - EventBuildingSequences: Make run in CA, simplify suffix in naming
    • f66c2a47 - EventBuildingSequences: Remove commented, added short description for getPEBBuildSuffix

    Compare with previous version

    • Author Developer
      Resolved by Frank Winklmeier

      Hello, the current code runs, all tests successful, however I don't think this part is implemented right:

          def pebInfoWriterToolGenerator(chainDict):
              if isComponentAccumulatorCfg():
                  cfg = pebInfoWriterToolCfg(flags, chainDict['chainName'], eventBuildType)
                  tool = cfg.popPrivateTools()
              else:
                  with ConfigurableCABehavior():
                      cfg = pebInfoWriterToolCfg(flags, chainDict['chainName'], eventBuildType)
                      tool = conf2toConfigurable(cfg.popPrivateTools())
              appendCAtoAthena(cfg)
              return tool

      It's using legacy support (appendCAtoAthena) and I don't think we want to need to differentiate whether we're in CA menu or not. What I need is a function generator that returns the hypo tool for the MenuSequenceCA setting. Does pebInfoWriterToolCfg have to return a CA, or can we have it return just the tool? Else, how do I just merge/discard the CA?

      cheers,

      Claire

  • Tomasz Bold mentioned in merge request !63204 (closed)

    mentioned in merge request !63204 (closed)

  • Frank Winklmeier mentioned in merge request !63378 (merged)

    mentioned in merge request !63378 (merged)

  • Claire Antel added 799 commits

    added 799 commits

    • f66c2a47...f451982e - 792 commits from branch atlas:23.0
    • 4da44d39 - TMMT EventBuildingSeq: Migrating to CA and newJO, removing NoCAmigration...
    • dff532be - EventBuildingSequences: Avoiding cfg clashes by uniquely naming pebInputMaker and PEBInfoWriterAlg
    • 0b46acc0 - EventBuildingSequences: uniquely naming RoIs
    • bf83544c - GenerateMenuMT: Readd catch of TypeError exception with print to log and rethrow of exception
    • 47ed3123 - EventBuildingSequences: Make run in CA, simplify suffix in naming
    • 5f0022e4 - EventBuildingSequences: Remove commented, added short description for getPEBBuildSuffix
    • d751e110 - EventBuildingSequences pebInfoWriterToolGenerator: Return CA if newJO, else tool if oldJO

    Compare with previous version

  • Jenkins please retry a build

  • This merge request affects 1 package:

    • Trigger/TriggerCommon/TriggerMenuMT

    This merge request affects 3 files:

    • Trigger/TriggerCommon/TriggerMenuMT/python/HLT/CommonSequences/EventBuildingSequences.py
    • Trigger/TriggerCommon/TriggerMenuMT/python/HLT/CommonSequences/TLABuildingSequences.py
    • Trigger/TriggerCommon/TriggerMenuMT/python/HLT/Config/GenerateMenuMT.py

    Adding @sutt ,@shanisch ,@fpastore ,@lidiaz as watchers

  • :pencil: :scissors: The system determined that CI tests (with names matching "^CITest_SimulationRun(2|3)(FullSim|Hit).*$") are not needed for this code change. They are not run. This is not an indicator to restart the job.

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading