Skip to content
Snippets Groups Projects

Reduce processing time for the jet hypo in the presence of identical conditions

Merged Reduce processing time for the jet hypo in the presence of identical conditions
peter/athena:master-ATR22238_2_1 into master
Merged Peter Sherwood requested to merge peter/athena:master-ATR22238_2_1 into master

More work, mostly configuration to reduce long running times in the presence of identical conditions.In particular the 10j40 chain had very long running times.

The issue was first brought up in ATR-22145. An initial fix was provided in MR !37225 (merged). This MR dealt with events where it could be easily determined that the event would fail. The tests do not contain events that pass the problematic chains.

This work goes on to address the long times that would occur in passing events.

@bossio @valentem @peter

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    This merge request affects 24 files. Since this is a long list, I will not print it here.

  • ATLAS Robot added JetEtmiss Trigger master review-pending-level-1 labels

    added JetEtmiss Trigger master review-pending-level-1 labels

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    :negative_squared_cross_mark: CI Result FAILURE (hash ad3a635f)

    Athena AthSimulation AthGeneration AnalysisBase
    externals :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    make :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    required tests :o: :white_check_mark: :white_check_mark: :white_check_mark:
    optional tests :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view
    :white_check_mark: Athena: number of compilation errors 0, warnings 0
    :white_check_mark: AthSimulation: number of compilation errors 0, warnings 0
    :white_check_mark: AthGeneration: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23371]

  • Gordon Watts
    Gordon Watts @gwatts ·
    Developer

    Seeing lots of errors that look like this (in the trigger test):

    ComboHypo_merged_EmptyMuonSeq2_NFSmuE...            INFO -- HLT_mu20_2mu4noL1_L1MU20 multiplicities: [1, 2] 
HLTAllSteps                                        ERROR AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<Gaudi::Sequence>]): code FAILURE: PBASE::sysInitialize() 
2020-Nov-13 00:30:39,365 ERROR [ at HLTAllSteps:-1775395873] HLTAllSteps: AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<Gaudi::Sequence>]): code FAILURE: PBASE::sysInitialize() 
HLTTop                                             ERROR Unable to initialize Algorithm AthSequencer/HLTAllSteps
  • Gordon Watts added review-user-action-required label and removed review-pending-level-1 label

    added review-user-action-required label and removed review-pending-level-1 label

  • Peter Sherwood added 1 commit

    added 1 commit

    • 7ba867cf - set jet hypo python debug flag to False

    Compare with previous version

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    This merge request affects 24 files. Since this is a long list, I will not print it here.

  • ATLAS Robot added review-pending-level-1 label and removed review-user-action-required label

    added review-pending-level-1 label and removed review-user-action-required label

  • Rafal Bielski
    Rafal Bielski @rbielski ·

    Hi @peter, it seems you pushed some Apple metadata files into this branch (the ._ files). Could you remove them?

  • Peter Sherwood added 1 commit

    added 1 commit

    • 8fc33924 - remove OSX metafiles, add to the documentation files

    Compare with previous version

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    This merge request affects 26 files. Since this is a long list, I will not print it here.

  • Rafal Bielski
    Rafal Bielski @rbielski ·

    Thanks!

  • Rafal Bielski mentioned in merge request !38211 (merged)

    mentioned in merge request !38211 (merged)

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    :negative_squared_cross_mark: CI Result FAILURE (hash 7ba867cf)

    Athena AthSimulation AthGeneration AnalysisBase
    externals :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    make :o: :white_check_mark: :white_check_mark: :o:
    required tests :o: :white_check_mark: :white_check_mark: :white_check_mark:
    optional tests :cloud: :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view
    :o: Athena: number of compilation errors 1, warnings 6
    :white_check_mark: AthSimulation: number of compilation errors 0, warnings 0
    :white_check_mark: AthGeneration: number of compilation errors 0, warnings 0
    :o: AnalysisBase: number of compilation errors 1, warnings 6
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23398]

  • P
    Peter Sherwood @peter ·
    Author Developer

    The compilation errors appear to be unrelated to this MR.

    For information - how is this possible? Is this MR mixed with development elsewhere?

    thanks

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    :negative_squared_cross_mark: CI Result FAILURE (hash 8fc33924)

    Athena AthSimulation AthGeneration AnalysisBase
    externals :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    make :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    required tests :o: :white_check_mark: :white_check_mark: :white_check_mark:
    optional tests :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view
    :white_check_mark: Athena: number of compilation errors 0, warnings 0
    :white_check_mark: AthSimulation: number of compilation errors 0, warnings 0
    :white_check_mark: AthGeneration: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 23406]

  • Rafal Bielski
    Rafal Bielski @rbielski ·

    Indeed I also don't know how the compilation failure in the previous pipeline was possible. But it looks like the test failures in the most recent pipeline are related to this MR:

    TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   FATAL  Exception with tag=TrigJetConditionConfig_et/et_0_fn retrieve is caught
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR TrigJetConditionConfig_et/et_0_fn retrieve 	 unable to dcast AlgTool TrigJetConditionConfig_et/et_0_fn to interface ITrigJetCapacityCheckedConditionConfig	 StatusCode=FAILURE
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR /var/lib/jenkins/workspace/CI-MERGE-REQUEST-CC7/master/Control/AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<AlgTool>]): code FAILURE: PBASE::sysInitialize()
ToolSvc                                        0   ERROR Error initializing tool 'TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_j45_L1J20.helper_0_fn_n1p0.fastreduction_0_fn.capacitychecked_0_fn'
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   FATAL  Exception with tag=PrivateTool retrieve is caught
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR PrivateTool retrieve 	 Failed to retrieve PrivateTool: TrigJetConditionConfig_capacitychecked/capacitychecked_0_fn	 StatusCode=FAILURE
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR /var/lib/jenkins/workspace/CI-MERGE-REQUEST-CC7/master/Control/AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<AlgTool>]): code FAILURE: PBASE::sysInitialize()
ToolSvc                                        0   ERROR Error initializing tool 'TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_j45_L1J20.helper_0_fn_n1p0.fastreduction_0_fn'
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   FATAL  Exception with tag=PrivateTool retrieve is caught
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR PrivateTool retrieve 	 Failed to retrieve PrivateTool: TrigJetHypoToolConfig_fastreduction/fastreduction_0_fn	 StatusCode=FAILURE
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR /var/lib/jenkins/workspace/CI-MERGE-REQUEST-CC7/master/Control/AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<AlgTool>]): code FAILURE: PBASE::sysInitialize()
ToolSvc                                        0   ERROR Error initializing tool 'TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_j45_L1J20.helper_0_fn_n1p0'
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   FATAL  Exception with tag=PrivateTool retrieve is caught
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR PrivateTool retrieve 	 Failed to retrieve PrivateTool: TrigJetHypoToolHelperMT/helper_0_fn_n1p0	 StatusCode=FAILURE
TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_...       0   ERROR /var/lib/jenkins/workspace/CI-MERGE-REQUEST-CC7/master/Control/AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<AlgTool>]): code FAILURE: PBASE::sysInitialize()
ToolSvc                                        0   ERROR Error initializing tool 'TrigJetHypoAlgMT_a4tcem_subjesIS.HLT_j45_L1J20'
TrigJetHypoAlgMT_a4tcem_subjesIS               0   ERROR TrigHLTJetHypo/src/TrigJetHypoAlgMT.cxx:23 (StatusCode TrigJetHypoAlgMT::initialize()): code FAILURE: m_hypoTools.retrieve()
TrigJetHypoAlgMT_a4tcem_subjesIS               0   ERROR /var/lib/jenkins/workspace/CI-MERGE-REQUEST-CC7/master/Control/AthenaBaseComps/AthenaBaseComps/AthCommonDataStore.icc:43 (StatusCode AthCommonDataStore<PBASE>::sysInitialize() [with PBASE = AthCommonMsg<Gaudi::Algorithm>]): code FAILURE: PBASE::sysInitialize()
TrigJetHypoAlgMT_a4tcem_subjesIS               0   ERROR DecisionHandling/src/HypoBase.cxx:25 (StatusCode HypoBase::sysInitialize()): code FAILURE: AthReentrantAlgorithm::sysInitialize()
Jet_view                                       0   ERROR Unable to initialize Algorithm TrigJetHypoAlgMT/TrigJetHypoAlgMT_a4tcem_subjesIS
  • P
    Peter Sherwood @peter ·
    Author Developer

    Thanks for the clarification!

  • Damian Gil added review-user-action-required label and removed review-pending-level-1 label

    added review-user-action-required label and removed review-pending-level-1 label

  • Peter Sherwood added 342 commits

    added 342 commits

    • 8fc33924...763fcf3e - 339 commits from branch atlas:master
    • 72bff91b - Merge tag 'nightly/master/2020-11-15T2101' into master-ATR22238_2_1
    • 93fb2af0 - Jet Trig Hypo update presentations in the doc directory
    • 37bd6406 - Contuiing work on optimisation for identical Conditions

    Compare with previous version

    Toggle commit list
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply