athena merge requestshttps://gitlab.cern.ch/atlas/athena/-/merge_requests2024-03-28T11:30:32+01:00https://gitlab.cern.ch/atlas/athena/-/merge_requests/70158athena.py: allow --config-only without storing config (legacy)2024-03-28T11:30:32+01:00Frank Winklmeierathena.py: allow --config-only without storing config (legacy)For legacy athena, allow the use of `--config-only` without specifying
a filename to make it compatible with CA-mode.
Also fix a bug introduced in !70102 when using `--config-only`.For legacy athena, allow the use of `--config-only` without specifying
a filename to make it compatible with CA-mode.
Also fix a bug introduced in !70102 when using `--config-only`.https://gitlab.cern.ch/atlas/athena/-/merge_requests/70184Athena: Temporarily remove MuonGeoModelTest and MuonGeoModelTestR4 from build2024-03-28T11:29:31+01:00Edward MoyseAthena: Temporarily remove MuonGeoModelTest and MuonGeoModelTestR4 from buildAs discussed in ATLASRECTS-7946, we will temporarily remove these packages from Athena since they are blocking CI from running.
(AthSimulation does not include them).
cc @fwinkl @jojunggeAs discussed in ATLASRECTS-7946, we will temporarily remove these packages from Athena since they are blocking CI from running.
(AthSimulation does not include them).
cc @fwinkl @jojunggehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/52633Introduce flat container to replace the CaloClusterROI collection.2024-03-28T11:20:08+01:00Goetz GayckenIntroduce flat container to replace the CaloClusterROI collection.Originally there was the CaloClusterROI_Selector which creates CaloClusterROIs
from selected calorimeter clusters. These objects are used in several algorithms.
However none of the algorithms uses these CaloClusterROI objects directly. ...Originally there was the CaloClusterROI_Selector which creates CaloClusterROIs
from selected calorimeter clusters. These objects are used in several algorithms.
However none of the algorithms uses these CaloClusterROI objects directly. Most
algorithms create flat vectors from the CaloClusterROI_Collection. The latter
is done once per event which leads to lock contention in the InDetDenseEnvAmbiTrackSelectionTool.
This MR replace the CaloClusterROIs by something that can be directly used by
the client algorithms. This avoids executing the same coordinate transformations
and creation of the flat vectors at several places, and mitigates the lock
contention (there is still a ReadHandle being created by the
InDetDenseEnvAmbiTrackSelectionTool for each track).
Lock contention in the InDetDenseEnvAmbiTrackSelectionTool is significantly reduced (at
large number of threads). And there seems to be a small overall improvement of 1-2%.
However the latter may not be significant.https://gitlab.cern.ch/atlas/athena/-/merge_requests/67505Draft: Update TruthIncidents and ISFParticle to know about uniqueID2024-03-28T11:19:33+01:00John Derek ChapmanDraft: Update TruthIncidents and ISFParticle to know about uniqueID(ATLASSIM-6978)(ATLASSIM-6978)https://gitlab.cern.ch/atlas/athena/-/merge_requests/69798Update MergeConfigs.py - new feature to select specific folders for config...2024-03-28T11:19:06+01:00Will ButtingerUpdate MergeConfigs.py - new feature to select specific folders for config...Update MergeConfigs.py - new feature to select specific folders for config merging (useful for running tests on subset of the tier0 monitoring)
Also adds a fix for the case where the hash of the release cannot be determined (as is the c...Update MergeConfigs.py - new feature to select specific folders for config merging (useful for running tests on subset of the tier0 monitoring)
Also adds a fix for the case where the hash of the release cannot be determined (as is the case if the user has setup their own cmake project)https://gitlab.cern.ch/atlas/athena/-/merge_requests/70133Added selection decoration block config for CP algs2024-03-28T11:18:54+01:00Thomas StreblerAdded selection decoration block config for CP algsFollow up to !69110, which moved the `AsgSelectionAlg` run at the end of the object working point configs to schedule them as part of the `OutputAnalysis` config block. In order to allow for analysis-specific algorithms to be inserted be...Follow up to !69110, which moved the `AsgSelectionAlg` run at the end of the object working point configs to schedule them as part of the `OutputAnalysis` config block. In order to allow for analysis-specific algorithms to be inserted before the `OutputAnalysis` config and rely on the decorations saved by those algorithms, a dedicated `SelectionDecoration` block is added, to be run before the analysis-specific algorithms.
FYI @omajersk @krumnackhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/70152Draft: barcode -> ID migration for HGTD/InDet/MuonDigitization (2nd attempt)2024-03-28T11:16:53+01:00John Derek ChapmanDraft: barcode -> ID migration for HGTD/InDet/MuonDigitization (2nd attempt)The first attempt at this !69655 caused issues in the case that pile-up background HITS with intact truth information and using a barcode-based persistent were read into a pile-up digitization job. In this case there was no way for the i...The first attempt at this !69655 caused issues in the case that pile-up background HITS with intact truth information and using a barcode-based persistent were read into a pile-up digitization job. In this case there was no way for the initial HepMcParticleLink member variables of the transient hit classes generated from the persistent classes to know whether they should be linking to an event other than the first event in the McEventCollection (The hard-scatter GenEvent is always first in the McEventCollection). This meant that calls to `HepMcParticleLink::id()` would return zero as the links often appear to be invalid.
This second version of the migration is less aggressive as it allows the possibility of barcode-based inputs. It avoids the issue seen in ATLSWUPGR-242 according to my local tests, but I will avoid marking the merge request as ready until we have a CI test to catch such an issue.https://gitlab.cern.ch/atlas/athena/-/merge_requests/70185AthConfigFlags: improve flags unit test2024-03-28T11:02:21+01:00Frank WinklmeierAthConfigFlags: improve flags unit test- use the full `AllConfigFlags` for testing command line parsing
- use a minimal parser to test `--help` message and ignore whitespace- use the full `AllConfigFlags` for testing command line parsing
- use a minimal parser to test `--help` message and ignore whitespacehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/701292024-03-26: merge of 24.0 into main2024-03-28T10:55:02+01:00Edward Moyse2024-03-26: merge of 24.0 into mainThis sweep contains the following MRs:
* !70090 ATR-28985 Trigger request for Multijet TLA ~Trigger, ~TriggerMenu
* !70113 MuonDQA: remove references to Run-2 monitoring config ~DQ, ~MuonSpectrometer
* !70115 LArMonitoring: delete leg...This sweep contains the following MRs:
* !70090 ATR-28985 Trigger request for Multijet TLA ~Trigger, ~TriggerMenu
* !70113 MuonDQA: remove references to Run-2 monitoring config ~DQ, ~MuonSpectrometer
* !70115 LArMonitoring: delete legacy job options ~DQ, ~LAr
* !70117 Pixel Calibration - adding getters and fixing typos ~InnerDetector
* !70112 ZdcMonitoring: cleanup references to legacy config ~ForwardDetectors
* !70109 Fix CMake warning in AthSimulation builds. ~Build
* !69914 Bug fix to generation of links between neutral FE and muons ~JetEtmiss, ~Reconstruction, ~Run2-DataReco-output-changed, ~Run2-MCReco-output-changed, ~Run3-DataReco-output-changed, ~Run3-MCReco-output-changed, ~Tools, ~frozen-tier0-violating
* !70100 Added maxParallelismExtra setting to AvalancheSchedulerSvcCfg ~Trigger
* !70096 Avoid duplicate streamer chain blocks ~Trigger, ~TriggerMenu
* !70108 Remove nullptr initialiser that causes a crashing ~Simulation
* !70048 Adding more test chains for delayed jets ~Trigger, ~TriggerMenu, ~changes-trigger-counts
* !70102 AthenaCommon: refactor --profile-python option ~Core
* !70097 remove legacy python config files form Calorimeter/CaloCnv ~Calorimeter
* !70095 ZDC reconstruction ART script, testing on ZDCCalib data ~ForwardDetectors
* !70092 remove obsolete config files from LArEventTest ~LAr
* !70091 remove obsolete config files from LArCondAthenaPool and LArDetMgrDetDescrCnv ~LAr
* !70089 remove obsolete config-files from LArByteStream ~LAr, ~Simulation
* !70074 AFP monitoring update code style ~DQ, ~ForwardDetectors
* !70069 Fix for the multi-threaded event service configuration ~Tools
* !70036 sTGC raw data monitoring: fix problem with shifter occupancy plots ~DQ, ~MuonSpectrometer
* !70060 Update eFexByteStreamTool.cxx - write multislice tobs as xtobs (since only... ~L1Calo, ~Trigger
* !70057 L1calo monitoring improvements ~Core, ~DQ, ~L1Calo, ~Trigger
* !70035 Update gfex parameters in trigger menu ~L1Calo, ~Trigger, ~TriggerMenu, ~changes-trigger-counts
* !69759 Monitoring of alternative L1 eTAU simulation ~DQ, ~Tau, ~Trigger
* !70030 ATR-28968: Reduce INFO messages from central trigger configuration code ~ACTS, ~Core, ~Tracking, ~Trigger
* !70011 fix failing DataQualityConfigurations tests in dbg build ~DQ
* !70045 CaloSamplingFractionAnalysis: delete legacy job options ~Simulation
* !69994 offline DQ for MMG trigger ~DQ, ~MuonSpectrometer
* !70073 fix StepName ~JetEtmiss, ~Trigger, ~TriggerJet, ~TriggerMenu
* !70061 IDTrig: do not switch on TRT with ITk, deal with InDet/ITk eta dependant cuts ~Trigger, ~TriggerID
* !70068 Run the TIDA Tier0 monitoring correctly ~Trigger
* !70042 pT filter for groomed jets before moment calculation ~JetEtmiss, ~Reconstruction
* !70044 adding missing et conversion factor for KFMET definition for Ph1Topo ~Trigger, ~TriggerMenu
* !70046 DumpGeo: delete legacy job options ~Geometry
* !69953 New efficiency DQ criteria for HLTmuon hanconfig ~DQ
* !70022 clean out legacy config code from LArCalorimeter/LArClusterRec ~LArhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69914Bug fix to generation of links between neutral FE and muons2024-03-28T10:55:02+01:00Mark HodgkinsonBug fix to generation of links between neutral FE and muonsA bug was found in the generation of these links, which was caused by an iterator in a double loop not being reset to begin().
The fix causes differences in:
JetETMissNeutralParticleFlowObjectsAuxDyn.FE_efrac_matched_muon
JetETMissNeu...A bug was found in the generation of these links, which was caused by an iterator in a double loop not being reset to begin().
The fix causes differences in:
JetETMissNeutralParticleFlowObjectsAuxDyn.FE_efrac_matched_muon
JetETMissNeutralParticleFlowObjectsAuxDyn.FE_nMatchedMuons
which does not lead to visible changes on MET, which is the only physics quantity currently using these links:
https://markhodgkinson.web.cern.ch/web_met_main_March18/index.html
At ESD level differences in the MET map are seen though, but these must lead to shifts which are smaller than the bin sizes in the above plots.
This MR also adds a new decoration onto charged FE, which is required for global pflow studies related to muons. This adds:
JetETMissChargedParticleFlowObjectsAuxDyn.FE_efrac_matched_muonhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69664Draft: athena.py: implement native support for CA2024-03-28T10:37:33+01:00Frank WinklmeierDraft: athena.py: implement native support for CAThis MR is a refactor of the functionality of `ThinCAWrapper.sh` and `CARunner.py` into the main `athena` executable. After the initial shell-phase, the process remains in python. This is achieved using the `runpy` module to directly lau...This MR is a refactor of the functionality of `ThinCAWrapper.sh` and `CARunner.py` into the main `athena` executable. After the initial shell-phase, the process remains in python. This is achieved using the `runpy` module to directly launch the CA script via the `__main__` entry point.
This MR is the first step in making the `--CA` option obsolete.
#### User visible changes:
* The interactive mode now requires an optional \<stage\> argument also for legacy athena. If this is directly followed by the job options, the arguments need to be terminated by `--`. E.g all of the following are equivalent:
```plaintext
> athena AthExHelloWorld/HelloWorldOptions.py -i
> athena -i -- AthExHelloWorld/HelloWorldOptions.py
> athena -i init AthExHelloWorld/HelloWorldOptions.py
```
* If the user tries to run a CA script in legacy mode, the following will be printed:
```plaintext
AthenaCommon.Include.IncludeError: include file AthExHelloWorld/HelloWorldConfig.py can not be found
If the script is a ComponentAccumulator configuration, add a shebang (#!) to its first line or run with the --CA option.
```
#### Implementation details:
* Use the `AthOptionsParser` for both legacy athena and CA-based job/athena. If used within `athena.py` the athena-specific command line options are added.
* The shell-based CLI-parsing is now restricted to what is necessary to setup the preloading.
* Implement the file path resolution in Python using `unixtools.FindFile`. The "shebang" CA detection now also works for file paths that are resolved at runtime.
* The `CARunner.py` use-case is replaced by directly loading the pkl file (legacy/CA pickles are automatically distinguished). I dropped the support of CA pkls without MainServicesCfg. Could be added again if really needed.
* Migrate the `--tracelevel` feature into `AthenaCommon.Debugging`. We should probably unify this with the existing `--trace` option in a second iteration.
For the moment, we still support the `--CA` flag. But if the proper shebang is used, the following commands all work transparently without the need to use `--CA`:
```sh
athena.py AthExHelloWorld/HelloWorldOptions.py
athena.py --config-only legacy.pkl AthExHelloWorld/HelloWorldOptions.py
athena.py legacy.pkl
athena.py AthExHelloWorld/HelloWorldConfig.py
athena.py --config-only cfg.pkl AthExHelloWorld/HelloWorldConfig.py
athena.py cfg.pkl
```
cc @wlampl @tadej @ssnyder
TODO:
* [ ] Delete `CARunner.py` after migrating the few uses to `athena.py`
* [ ] Backport to ~"24.0" once validated in ~mainhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/70141MuonTesterTree --- Use MSG macros and inherit from AthMessasing2024-03-28T09:37:43+01:00Johannes Junggeburthjohannes.josef.junggeburth@cern.chMuonTesterTree --- Use MSG macros and inherit from AthMessasinghttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69732Draft: Pin the NCB segment onto th calo exit2024-03-28T08:23:27+01:00Johannes Junggeburthjohannes.josef.junggeburth@cern.chDraft: Pin the NCB segment onto th calo exitFirst attempt to reduce the beam background by adding a new constraint to the NCB segment. Introduce the additional constraint that the particle is roughly horizontal and crossed the calorimeter at z=0 roughly at the same height. This co...First attempt to reduce the beam background by adding a new constraint to the NCB segment. Introduce the additional constraint that the particle is roughly horizontal and crossed the calorimeter at z=0 roughly at the same height. This constaint drastically reduces the number of built NCB segments (cf. plots below - back is the current release and red are the patched distributions). Plots have been produced using one file from the heavy ion data-taking.
![image](/uploads/c46e78ee3a9769a62ec386694afbfa57/image.png)
![image](/uploads/854f83055693012100e031b2f38b1f55/image.png)
Tagging: @wlampl, @ekay, @pscholer, @calpigia, @verduccihttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69314R4 sTGC Geometry - Adding Pad Readout Element2024-03-28T07:37:46+01:00Ishan Kiritbhai VyasR4 sTGC Geometry - Adding Pad Readout ElementAdding Pads to sTgc similar to wireGroup and Strips.
@alainb @jojungge @pscholerAdding Pads to sTgc similar to wireGroup and Strips.
@alainb @jojungge @pscholerhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69113Draft: 1D Hough and more FPGATrackSim fun!2024-03-27T22:28:01+01:00Jahred Adelmanjahreda@gmail.comDraft: 1D Hough and more FPGATrackSim fun!I ported over the latest version of the 1D Hough from Gretel and 21.9. I also updated HTTSectorBank to know about q/pt binning. A few things, mostly questions for @brosser :
1) The EtaPatternTool we have is quite old, too. Should I take ...I ported over the latest version of the 1D Hough from Gretel and 21.9. I also updated HTTSectorBank to know about q/pt binning. A few things, mostly questions for @brosser :
1) The EtaPatternTool we have is quite old, too. Should I take the latest one from 21.9?
2) To be honest it's still not clear to me how the q/pt binning should be transmitted to the sectors since it's not hard-coded in the file. We have it as a map tag option, but it's not obvious to me how that should work since it is defined at matrix generation. So I may still be missing something things in the matrix generation and/or downstream.
3) I removed the htt namepsace everywhere :smile:
Tagging @tbold as well.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69961Add MC Hits to AOD2024-03-27T21:30:20+01:00Mateus Hufnagelmateus.hufnagel@cern.chAdd MC Hits to AODThis MR is to include:
* At `LArClusterCellDumper/CaloThinCellsInAODAlgConfig` adds `LArEMBHits` mapped from `Clusters` region, and writes to AOD.
* Some error treatment at `EventReaderAlg` init.
* Input containers at `EventReaderAlg` r...This MR is to include:
* At `LArClusterCellDumper/CaloThinCellsInAODAlgConfig` adds `LArEMBHits` mapped from `Clusters` region, and writes to AOD.
* Some error treatment at `EventReaderAlg` init.
* Input containers at `EventReaderAlg` renamed to match AOD `CaloThinCellsInAODAlg` outputs.
This development is in the context of the QT of @mhufnage, related to cross-talk mitigation studies at LAr Calorimeter.
ping @gunal, @laforge, @pavol, @eegidiop, @lucianomMateus Hufnagelmateus.hufnagel@cern.chMateus Hufnagelmateus.hufnagel@cern.chhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69110Ability to aggregate all selection flags into single output variable in Outpu...2024-03-27T19:47:17+01:00Oliver MajerskyAbility to aggregate all selection flags into single output variable in OutputAnalysisConfigOne of the "inconveniences" of CP algorithms at the moment is that we end up with several selection flags per object in the output, inflating the output size and also potentially increasing user error.
Typical scenario:
For jets, we hav...One of the "inconveniences" of CP algorithms at the moment is that we end up with several selection flags per object in the output, inflating the output size and also potentially increasing user error.
Typical scenario:
For jets, we have jet_select_jvt, which contains just the JVT selection, and then jet_select_or, which contains overlap removal decision.
For electrons, if we have a `tight` working point defined, we have `el_select_tight` which contains the lepton pre-selection + the working point selection, and then `el_select_or` which contains overlap removal decision.
In this MR a proposal for solving this is to add in the output block a logic to pick up for each output container all selections (so-called working points) and store a single flag agggregating all pre-selections, working point selection, and also overlap removal, if enabled.
Example of what happened **before this MR** with electron selection flag `el_select_tight`:
```
| /*** PythonConfig AnaAlgorithm/CP::AsgSelectionAlg/ElectronSelectionSummary_tight
| |- particles: 'AnaElectrons_%SYS%'
| |- preselection: 'selectEta,as_bits&&trackSelection,as_bits&&goodOQ,as_bits&&selectPt,as_bits&&selectLikelihood_tight,as_bits&&isolated_tight,as_bits'
| |- selectionDecoration: 'baselineSelection_tight,as_char'
| \--- (End of PythonConfig AnaAlgorithm/CP::AsgSelectionAlg/ElectronSelectionSummary_tight)
```
-> no OR flag included, even if we enable adding OR to each selection (see below). Also the PtEtaSelection block is not propagated, even though it is added in the block config!
Example of what happens **after this MR** with electron selection flag `el_select_tight`:
```
| /*** PythonConfig AnaAlgorithm/CP::AsgSelectionAlg/ObjectSelectionSummary_AnaElectrons_tight
| |- particles: 'AnaElectrons_%SYS%'
| |- preselection: 'selectEta,as_bits&&trackSelection,as_bits&&goodOQ,as_bits&&selectPt,as_bits&&selectLikelihood_tight,as_bits&&isolated_tight,as_bits&&selectPtEta&&passesOR_%SYS%,as_char'
| |- selectionDecoration: 'baselineSelection_tight'
| \--- (End of PythonConfig AnaAlgorithm/CP::AsgSelectionAlg/ObjectSelectionSummary_AnaElectrons_tight)
```
-> OR flag added when enabled + also PtEtaSelection correctly propagated.
There is also a switch added to disable making these flags in the output, if not desired (e.g. if an analysis only cares about making some loosely-preselected NTuple and manages all further selections offline).
In addition, I introduce and set by default a new config option `addToAllSelections` for [OverlapAnalysisConfig](https://acode-browser1.usatlas.bnl.gov/lxr/source/athena/PhysicsAnalysis/Algorithms/AsgAnalysisAlgorithms/python/OverlapAnalysisConfig.py) block. If this is true, the OR decision is added by default to all selections above, which is the default behavior that we want. We already see that users fail to understand this, and assume, that e.g. passing `AnaElectrons.tight` to event selection means, that event selection is applied using tight electrons including overlap removal, but this is not the case by default. To revert to the old behavior where OR decision is decoupled from selections, one just needs to switch `addToAllSelection` to `False`.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69929Draft: Migrate EgammaAnalysisAlgorithms test to block config2024-03-27T19:38:02+01:00Thomas StreblerDraft: Migrate EgammaAnalysisAlgorithms test to block configOther CP algorithms to followOther CP algorithms to followhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69962CP algs: Move main tests to CA 242024-03-27T19:37:34+01:00Thomas StreblerCP algs: Move main tests to CA 24Cherry-pick of !69307 for ~"24.0"
Needed to convert old sequence to CA
FYI @tadej @krumnackCherry-pick of !69307 for ~"24.0"
Needed to convert old sequence to CA
FYI @tadej @krumnackhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/70101Fix FPGASim workflow test to use input RDO from CVMFS2024-03-27T19:37:25+01:00Tomasz BoldFix FPGASim workflow test to use input RDO from CVMFSCloses EFTRACK-504
This MR improves FPGASim testing scriptCloses EFTRACK-504
This MR improves FPGASim testing script