Update L/M/T cuts for eEM threshold [ATR-27067] and implement checks of nesting [ATR-27796]

As a consequence of !63577 (merged) implementing the first iteration of ATR-27067, we realised (ATR-27796) that L1Calo PID thresholds not obeying a strict ordering in every eta bin can cause unintuitive behaviour. Notably, modifying only the Tight cut with reta being looser than Medium in specific eta bins resulted in L1CaloSim/FW differences leaking into the Medium definition:

• If an event passes Tight, L1CaloSim concludes that Mediumis passed, hence looseningTightregionally caused count increases inMedium`
• If an event fails Medium, the FW would conclude that Tight is failed, hence candidates could fail Tight despite passing the numerical thresholds -- opposite to what L1CaloSim would do

To avoid this, @watsona suggested to enforce that T >= M >= L is satisfied for each variable's working points in every eta bin when the thresholds are defined in the menu. This check fails as follows for the problematic cuts when running generateL1MenuRun3.py PhysicsP1_pp_run3_v1:

Py:TriggerMenuMT:TypeWideThresholdConfig   ERROR Medium_reta_fw (92) > Tight_reta_fw (86) for eta bin -25
Traceback (most recent call last):
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/bin/generateL1MenuRun3.py", line 44, in <module>
    sys.exit( main() )
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/bin/generateL1MenuRun3.py", line 39, in main
    generateL1Menu(flags)
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/23.0_Athena_x86_64-centos7-gcc11-opt/2023-06-22T2101/Athena/23.0.34/InstallArea/x86_64-centos7-gcc11-opt/python/TrigConfigSvc/TrigConfigSvcCfg.py", line 200, in generateL1Menu
    l1cfg = L1MenuConfig(
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/L1MenuConfig.py", line 73, in __init__
    self._generate()
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/L1MenuConfig.py", line 90, in _generate
    self._generateMenu()
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/L1MenuConfig.py", line 735, in _generateMenu
    self.l1menu.checkPtMinToTopo()
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/Base/L1Menu.py", line 379, in checkPtMinToTopo
    ttconfig = getTypeWideThresholdConfig(thrtype,self.do_HI_tob_thresholds)
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/Config/TypeWideThresholdConfig.py", line 92, in getTypeWideThresholdConfig
    return getConfig_eEM()
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/Config/TypeWideThresholdConfig.py", line 268, in getConfig_eEM
    validate_ordering(var,"Medium","Tight",confObj["workingPoints"])
  File "/lustre/fs22/group/atlas/tjkhoo/athena_dev/TMMT-alt/build/x86_64-centos7-gcc11-opt/python/TriggerMenuMT/L1/Config/TypeWideThresholdConfig.py", line 64, in validate_ordering
    assert leq_all_eta(_lesser,_greater), f"Working point ordering violated for {_lesser.name}, {_greater.name}"
AssertionError: Working point ordering violated for Medium_reta_fw, Tight_reta_fw

The check is preemptively implemented for jEM, jTAU, cTAU and eTAU as well.

The python check handles either both WPs being eta-dependent or both being eta-independent. Cross-comparisons could be implemented too, but as they are not currently necessary, such comparisons are identified as an error condition.

New eta-dependent eEM L,M,T cuts that satisfy the ordering principle were optimised by @thompson @hillier and are also included here.

The C++ implementation is done by defining comparison operators for the WorkingPoints_[thresholdtype] class, so that the enforcement check can in principle be applied to ValueWithEtaDependence<T> where T is a numerical type. This is chosen to evaluate as WP1 < WP2 if that holds for every value in all eta bins, which may not be entirely intuitive (in particular the behaviour of >, which is only true if this is satisfied in at least one eta bin, but < is satisfied in none) . Technically, only one (> or <) operator would need to be defined, but this would likely be more confusing. Note: this would have been much nicer in C++20 as it suffices only to overload <=>.

The keys used for the DoubleMenuLoad test referred to a L1 menu with eTAU cuts that did not obey the criteria above, so I have now switched this to check a recent ATN that predates !63577 (merged) (which would also fail). It hasn't been updated since !46711 (merged), so it doesn't hurt to have a more recent one. FYI @stelzer @mark in case either of you would prefer to choose different keys for whatever reason. We could generate the current menu and upload for these purposes, but I don't think it's all that necessary.

As pointed out by @mark, the C++ check on menu load is undesirable because it would break jobs that load older menus which violate the ordering.

@astruebi, @lidiaz

This merge request affects 2 packages:

• Trigger/TrigConfiguration/TrigConfData
• Trigger/TriggerCommon/TriggerMenuMT

Affected files list will not be printed in this case

Adding @sutt ,@jmharris ,@oliveirg ,@orlando ,@cmorenom ,@mark ,@tamartin ,@paulama ,@asonay ,@lidiaz as watchers

added 23.0 Trigger TriggerMenu analysis-review-required review-pending-level-1 labels

changed the description

The system determined that CI tests (with names matching "^CITest_SimulationRun(2|3)(FullSim|Hit).*$") are not needed for this code change. They are not run. This is not an indicator to restart the job.

CI Result FAILURE (hash 9be433db)

	Athena	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 73327]

added 1 commit

• acd64436 - Invert ordering logic for eTAU

Compare with previous version

This merge request affects 2 packages:

• Trigger/TrigConfiguration/TrigConfData
• Trigger/TriggerCommon/TriggerMenuMT

Affected files list will not be printed in this case

Adding @sutt ,@jmharris ,@oliveirg ,@orlando ,@cmorenom ,@mark ,@tamartin ,@paulama ,@asonay ,@lidiaz as watchers

The system determined that CI tests (with names matching "^CITest_SimulationRun(2|3)(FullSim|Hit).*$") are not needed for this code change. They are not run. This is not an indicator to restart the job.

CI Result FAILURE (hash acd64436)

	Athena	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 73328]

added 1 commit

• 9364d0dd - Update to a more recent, legal set of keys from ATN DB for double menu load test

Compare with previous version

This merge request affects 2 packages:

• Trigger/TrigConfiguration/TrigConfData
• Trigger/TriggerCommon/TriggerMenuMT

Affected files list will not be printed in this case

Adding @sutt ,@jmharris ,@oliveirg ,@orlando ,@cmorenom ,@mark ,@tamartin ,@paulama ,@asonay ,@lidiaz as watchers

changed the description

The system determined that CI tests (with names matching "^CITest_SimulationRun(2|3)(FullSim|Hit).*$") are not needed for this code change. They are not run. This is not an indicator to restart the job.

CI Result SUCCESS (hash 9364d0dd)

	Athena	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 73329]

Hi @khoo

Could you explain who you need the protection in the c++? Won't throwing a runtime error there will just break reco any time we want to run over older data with the incorrect conditions (or did we not record any in such a state)? This is partly why the double load keeps old keys to make sure no recent changes affect loading the first format.

Cheers, Mark

added 3 commits

• 8c6ce2e4 - Add python check for nesting of eEM cuts
• f087c71b - New LMT cuts for eEM
• 13203b85 - Add nesting python check for eTAU, jEM, jTAU, cTAU

Compare with previous version

This merge request affects 1 package:

• Trigger/TriggerCommon/TriggerMenuMT

This merge request affects 2 files:

• Trigger/TriggerCommon/TriggerMenuMT/python/L1/Config/FexThresholdParameters.py
• Trigger/TriggerCommon/TriggerMenuMT/python/L1/Config/TypeWideThresholdConfig.py

Adding @sutt ,@lidiaz as watchers

resolved all threads

The system determined that CI tests (with names matching "^CITest_SimulationRun(2|3)(FullSim|Hit).*$") are not needed for this code change. They are not run. This is not an indicator to restart the job.

CI Result SUCCESS (hash 13203b85)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 73334]