athena merge requestshttps://gitlab.cern.ch/atlas/athena/-/merge_requests2024-03-11T10:43:24+01:00https://gitlab.cern.ch/atlas/athena/-/merge_requests/69680Draft: Update SUSY_SimplifiedModel_PreInclude.py2024-03-11T10:43:24+01:00Javier Montejo BerlingenDraft: Update SUSY_SimplifiedModel_PreInclude.pyWrite mixing directly to param_blocks instead of returning a copy, which will anyway be used to update param_blocks.
Add some loggingWrite mixing directly to param_blocks instead of returning a copy, which will anyway be used to update param_blocks.
Add some logginghttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69664Draft: athena.py: implement native support for CA2024-03-28T10:37:33+01:00Frank WinklmeierDraft: athena.py: implement native support for CAThis MR is a refactor of the functionality of `ThinCAWrapper.sh` and `CARunner.py` into the main `athena` executable. After the initial shell-phase, the process remains in python. This is achieved using the `runpy` module to directly lau...This MR is a refactor of the functionality of `ThinCAWrapper.sh` and `CARunner.py` into the main `athena` executable. After the initial shell-phase, the process remains in python. This is achieved using the `runpy` module to directly launch the CA script via the `__main__` entry point.
This MR is the first step in making the `--CA` option obsolete.
#### User visible changes:
* The interactive mode now requires an optional \<stage\> argument also for legacy athena. If this is directly followed by the job options, the arguments need to be terminated by `--`. E.g all of the following are equivalent:
```plaintext
> athena AthExHelloWorld/HelloWorldOptions.py -i
> athena -i -- AthExHelloWorld/HelloWorldOptions.py
> athena -i init AthExHelloWorld/HelloWorldOptions.py
```
* If the user tries to run a CA script in legacy mode, the following will be printed:
```plaintext
AthenaCommon.Include.IncludeError: include file AthExHelloWorld/HelloWorldConfig.py can not be found
If the script is a ComponentAccumulator configuration, add a shebang (#!) to its first line or run with the --CA option.
```
#### Implementation details:
* Use the `AthOptionsParser` for both legacy athena and CA-based job/athena. If used within `athena.py` the athena-specific command line options are added.
* The shell-based CLI-parsing is now restricted to what is necessary to setup the preloading.
* Implement the file path resolution in Python using `unixtools.FindFile`. The "shebang" CA detection now also works for file paths that are resolved at runtime.
* The `CARunner.py` use-case is replaced by directly loading the pkl file (legacy/CA pickles are automatically distinguished). I dropped the support of CA pkls without MainServicesCfg. Could be added again if really needed.
* Migrate the `--tracelevel` feature into `AthenaCommon.Debugging`. We should probably unify this with the existing `--trace` option in a second iteration.
For the moment, we still support the `--CA` flag. But if the proper shebang is used, the following commands all work transparently without the need to use `--CA`:
```sh
athena.py AthExHelloWorld/HelloWorldOptions.py
athena.py --config-only legacy.pkl AthExHelloWorld/HelloWorldOptions.py
athena.py legacy.pkl
athena.py AthExHelloWorld/HelloWorldConfig.py
athena.py --config-only cfg.pkl AthExHelloWorld/HelloWorldConfig.py
athena.py cfg.pkl
```
cc @wlampl @tadej @ssnyder
TODO:
* [ ] Delete `CARunner.py` after migrating the few uses to `athena.py`
* [ ] Backport to ~"24.0" once validated in ~mainhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69620btagging tool for json-based CDI2024-03-29T09:41:06+01:00Binbin Dongbtagging tool for json-based CDItowards the JSON-based CDI by adding a selection tool to handle JSON-based configuration.towards the JSON-based CDI by adding a selection tool to handle JSON-based configuration.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69592Draft: Removal of H6 TB simu2024-03-08T08:59:33+01:00Pavol StrizenecDraft: Removal of H6 TB simuThis should remove H6 TB simulations, as agreed on https://its.cern.ch/jira/browse/ATLASSIM-7104This should remove H6 TB simulations, as agreed on https://its.cern.ch/jira/browse/ATLASSIM-7104https://gitlab.cern.ch/atlas/athena/-/merge_requests/69574Draft: adding Large D0 tracks to HIP and UPC HImodes2024-03-06T23:28:45+01:00Sebastian Tapia ArayaDraft: adding Large D0 tracks to HIP and UPC HImodesadding LargeD0 tacking pass to HImodes: HIP and UPCadding LargeD0 tacking pass to HImodes: HIP and UPChttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69569MetaReader: print warning when ignoring all but first entry from /TagInfo fields2024-03-11T14:57:18+01:00Maciej Pawel SzymanskiMetaReader: print warning when ignoring all but first entry from /TagInfo fieldsFollow-up from https://gitlab.cern.ch/atlas/athena/-/merge_requests/68982/ on request from @tstreble in [ATEAM-958](https://its.cern.ch/jira/browse/ATEAM-958?focusedId=6302527&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acommen...Follow-up from https://gitlab.cern.ch/atlas/athena/-/merge_requests/68982/ on request from @tstreble in [ATEAM-958](https://its.cern.ch/jira/browse/ATEAM-958?focusedId=6302527&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-6302527).https://gitlab.cern.ch/atlas/athena/-/merge_requests/69513Use config flags in analysis config2024-03-28T13:34:35+01:00Tadej Novaktadej.novak@cern.chUse config flags in analysis configMake config flags the main way to steer analysis config. This will help people working on analysis and Athena to feel at home and prevent accumulation of arguments of the `ConfigAccumulator`.
The naming is not fixed, I will probably als...Make config flags the main way to steer analysis config. This will help people working on analysis and Athena to feel at home and prevent accumulation of arguments of the `ConfigAccumulator`.
The naming is not fixed, I will probably also convert more items to flags. This is mainly to start the discussion and freeze the API as soon asp possible. I will probably then break this into multiple MRs.
Also some core flags should be renamed (see https://its.cern.ch/jira/browse/ATEAM-964).
Tagging @krumnack, @jolamber, @gwatts, @ekourlit for core AMG. Also tagging interested parties @ravinab, @omajersk, @tstreble, @khoo, @jchapman.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69417Draft: Remove Legacy to CA conversion utilities2024-03-08T12:37:40+01:00Tomasz BoldDraft: Remove Legacy to CA conversion utilitiesThis MR is a test to see how far we are from fully relying on CA job options.This MR is a test to see how far we are from fully relying on CA job options.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69403Draft: attempt of CA configuration optimization2024-03-08T14:25:11+01:00Francesca PastoreDraft: attempt of CA configuration optimizationhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69395Draft: update tags for NSW cluster uncertainties in data2024-03-12T21:41:53+01:00Patrick ScholerDraft: update tags for NSW cluster uncertainties in dataUpdating the tag for the NSW cluster uncertainties used in data. The updated tags will also go into the next global conditions tag (ATLGBLCONDTAGS-123). Currently, the folders are not yet present in a conditions tag, therefore they are h...Updating the tag for the NSW cluster uncertainties used in data. The updated tags will also go into the next global conditions tag (ATLGBLCONDTAGS-123). Currently, the folders are not yet present in a conditions tag, therefore they are hardcoded here.
Changes in the output of the data reco tests are expectedhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69382Adding new AthenaExamples package - AthExBasics2024-03-29T08:19:32+01:00James CatmoreAdding new AthenaExamples package - AthExBasicsThis MR adds a new package to AthenaExamples. It is intended to contain examples of tools, algorithms and config scripts for doing very basic real-world tasks such as reading the trigger decision, accessing and selecting xAOD objects, ru...This MR adds a new package to AthenaExamples. It is intended to contain examples of tools, algorithms and config scripts for doing very basic real-world tasks such as reading the trigger decision, accessing and selecting xAOD objects, running trigger matching &c. It is hoped developers will add to these over time.
Obviously this will have no impact on any existing workflows.
This is part of the lead-up to the documentation week in April.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69361Draft: add unit test reference files for DL1dv012024-03-01T10:18:21+01:00Ligang XiaDraft: add unit test reference files for DL1dv01- add unit test references for DL1dv01
- The motivation for this MR is to detect (avoid) the impact on using old tagger due to implementing new features, like https://gitlab.cern.ch/atlas/athena/-/merge_requests/68453.
tagging @bdong- add unit test references for DL1dv01
- The motivation for this MR is to detect (avoid) the impact on using old tagger due to implementing new features, like https://gitlab.cern.ch/atlas/athena/-/merge_requests/68453.
tagging @bdonghttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69343Draft: Adding Monitor Chain for the L1TopoSimulationOnlineMonitor2024-03-21T15:35:33+01:00Javier Jimenez PenaDraft: Adding Monitor Chain for the L1TopoSimulationOnlineMonitorI have added a chain for the online monitoring of L1TopoSimulation. The chain will include the L1TopoSimulation and subsequently, the L1TopoOnlineMonitor when the following flags are set to true: flags.Trigger.L1.doMuonTopoInputs and fla...I have added a chain for the online monitoring of L1TopoSimulation. The chain will include the L1TopoSimulation and subsequently, the L1TopoOnlineMonitor when the following flags are set to true: flags.Trigger.L1.doMuonTopoInputs and flags.Trigger.L1.doTopo
The inclusion of the chain is discussed in the following Jira tickets: https://its.cern.ch/jira/browse/ATR-27203 and https://its.cern.ch/jira/browse/ATR-27211.
The code included in this MR is based in the L1Topo legacy version of the same chain that was removed from athena in this MR: https://gitlab.cern.ch/atlas/athena/-/merge_requests/66681/
Any reference to L1TopoLegacy has been removed.Javier Jimenez PenaJavier Jimenez Penahttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69332Draft: Add WP support and SF for LRT muons2024-03-04T12:12:02+01:00Sagar AddepalliDraft: Add WP support and SF for LRT muonsThis MR adds a dedicated support for running MuonSelectionTool on LRT muons, and also adds efficiency SF and relevant systematics for LRT muons.This MR adds a dedicated support for running MuonSelectionTool on LRT muons, and also adds efficiency SF and relevant systematics for LRT muons.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69308Draft: Add AthXRT service and AthExXRT example (ATLINFR-5117)2024-03-08T10:57:10+01:00Quentin BerthetDraft: Add AthXRT service and AthExXRT example (ATLINFR-5117)This MR add the AthXRT service and AthExXRT example to Athena.
This service is a prototype to interact with AMD FPGA accelerators from Athena. It is quite basic and can only be used to enumerate and program the FPGA with a configuration...This MR add the AthXRT service and AthExXRT example to Athena.
This service is a prototype to interact with AMD FPGA accelerators from Athena. It is quite basic and can only be used to enumerate and program the FPGA with a configuration file (bitstream) at the moment. It is used for the exploration of EFCalo topoclustering FPGA acceleration, and might be useful for other Run4 FPGA acceleration prototypes (EFTracking have also used this code). The structure is heavily inspired from AthCUDA and AthExCUDA and is intended to be the common entry point for AMD FPGA accelerators in Athena.
The AthExXRT example illustrates how to use the AthXRT service to run a dummy vector addition kernel from Athena. An HLS code and build procedure is also provided to produce the required bitstream (Note that some public Apache 2.0 licensed code from AMD/Xilinx is used there).
This code requires the XRT library and it is currently not provided in any Atlas standard environments (See [this](https://its.cern.ch/jira/browse/ATLINFR-5117) ticket for the discussion on XRT inclusion in AtlasExternals). Because of this, these new packages are not compiled if XRT library is not available.
Note to reviewer: this is my first MR, and the first code that I ever written for Athena, so I expect modifications to be required before merging.Quentin BerthetQuentin Berthethttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69229Updating LAr EM NTuple creation to CA based configuration2024-03-28T18:07:37+01:00Mustafa Andre SchmidtUpdating LAr EM NTuple creation to CA based configurationUpdating configuration file and README file to switch NTuple creation from legacy to CA-based configuration.Updating configuration file and README file to switch NTuple creation from legacy to CA-based configuration.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69203Draft: HepMcParticleLink: Testing the adaption to GenParticle::id2024-02-29T14:08:28+01:00John Derek ChapmanDraft: HepMcParticleLink: Testing the adaption to GenParticle::idFollows !69076Follows !69076https://gitlab.cern.ch/atlas/athena/-/merge_requests/69157Draft: Remove wrong defined "RNNEleScoreSigTrans" variable (xAOD change)2024-02-27T08:10:03+01:00Antonio De MariaDraft: Remove wrong defined "RNNEleScoreSigTrans" variable (xAOD change)This MR is meant to remove "RNNEleScoreSigTrans" variable in xAODs since this is a variable derived from buggy flattening procedure. This should be substituted by "RNNEleScoreSigTrans_v1". The drop of "RNNEleScoreSigTrans" from PHYS and ...This MR is meant to remove "RNNEleScoreSigTrans" variable in xAODs since this is a variable derived from buggy flattening procedure. This should be substituted by "RNNEleScoreSigTrans_v1". The drop of "RNNEleScoreSigTrans" from PHYS and PHYSLITE was already done previously (https://gitlab.cern.ch/atlas/athena/-/merge_requests/66089), so the only change expect here is at xAOD level
Also, given this change, the following flags won't be defined anymore:
* IsTauFlag.EleRNNLoose
* IsTauFlag.EleRNNMedium
* IsTauFlag.EleRNNTighthttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69113Draft: 1D Hough and more FPGATrackSim fun!2024-03-27T22:28:01+01:00Jahred Adelmanjahreda@gmail.comDraft: 1D Hough and more FPGATrackSim fun!I ported over the latest version of the 1D Hough from Gretel and 21.9. I also updated HTTSectorBank to know about q/pt binning. A few things, mostly questions for @brosser :
1) The EtaPatternTool we have is quite old, too. Should I take ...I ported over the latest version of the 1D Hough from Gretel and 21.9. I also updated HTTSectorBank to know about q/pt binning. A few things, mostly questions for @brosser :
1) The EtaPatternTool we have is quite old, too. Should I take the latest one from 21.9?
2) To be honest it's still not clear to me how the q/pt binning should be transmitted to the sectors since it's not hard-coded in the file. We have it as a map tag option, but it's not obvious to me how that should work since it is defined at matrix generation. So I may still be missing something things in the matrix generation and/or downstream.
3) I removed the htt namepsace everywhere :smile:
Tagging @tbold as well.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69076Draft: HepMcParticleLink: Flag whether link points to suppressed pile-up truth2024-02-23T17:38:05+01:00John Derek ChapmanDraft: HepMcParticleLink: Flag whether link points to suppressed pile-up truthFollows !69055Follows !69055