athena merge requestshttps://gitlab.cern.ch/atlas/athena/-/merge_requests2023-12-21T13:05:12+01:00https://gitlab.cern.ch/atlas/athena/-/merge_requests/67944TriggerMenuMT: fix missing input maker for EmptyMenuSequenceCA2023-12-21T13:05:12+01:00Frank WinklmeierTriggerMenuMT: fix missing input maker for EmptyMenuSequenceCAConsistently use the `maker` property of `EmptyMenuSequenceCA` to make
sure the maker alg is retrieved correctly from the CA. Resolves
differences seen between running at INFO and DEBUG level.
Fixes ATR-28644, ATR-28615 and ATR-28477.
...Consistently use the `maker` property of `EmptyMenuSequenceCA` to make
sure the maker alg is retrieved correctly from the CA. Resolves
differences seen between running at INFO and DEBUG level.
Fixes ATR-28644, ATR-28615 and ATR-28477.
cc @fpastore @tamartin @smhhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/63369Feed Jet super Roi vertex to fast flavour tagging2023-06-05T16:45:13+02:00Thandikire MadulaFeed Jet super Roi vertex to fast flavour taggingMR to add GNN tagger utilising super roi vertexing into preselection motivated by the performance gain seen in the attached plot.
[ttbar_roc_btag.pdf](/uploads/9e15ec2b48d70555a661bd910b9b5931/ttbar_roc_btag.pdf)MR to add GNN tagger utilising super roi vertexing into preselection motivated by the performance gain seen in the attached plot.
[ttbar_roc_btag.pdf](/uploads/9e15ec2b48d70555a661bd910b9b5931/ttbar_roc_btag.pdf)https://gitlab.cern.ch/atlas/athena/-/merge_requests/69452Adding Scan WP chains to dev2024-03-06T19:09:12+01:00Lucas BezioAdding Scan WP chains to devAs mentionned in [ATR-28870](https://its.cern.ch/jira/browse/ATR-28870). A scan of WP needs to be done to understand the Rate-WP relation for fastDIPS+GN1 & fastGN2+GN2 configuration of the bjet tagging in HLT.
Tagging @lshi @lbeemste @...As mentionned in [ATR-28870](https://its.cern.ch/jira/browse/ATR-28870). A scan of WP needs to be done to understand the Rate-WP relation for fastDIPS+GN1 & fastGN2+GN2 configuration of the bjet tagging in HLT.
Tagging @lshi @lbeemste @cpollardhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/68731Fixing fastGN2 discriminant in preselection2024-02-13T03:00:53+01:00Lucas BezioFixing fastGN2 discriminant in preselectionfastGN2 was not changing counts. It was found that inference was done but not the discriminant computation. It was due to a regex not being updated in the original MR https://gitlab.cern.ch/atlas/athena/-/merge_requests/68273 and prevent...fastGN2 was not changing counts. It was found that inference was done but not the discriminant computation. It was due to a regex not being updated in the original MR https://gitlab.cern.ch/atlas/athena/-/merge_requests/68273 and preventing Bjet preselection to be applied for fastGN2 chains.
It is fixed and references are updated
Tagging @lshi @lbeemste @sfranchehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/68707Update TriggerConfigFlags.py - move flag inside Menu subcategory2024-02-09T16:06:42+01:00Will ButtingerUpdate TriggerConfigFlags.py - move flag inside Menu subcategoryWanted to make it clearer this flag will only affect menu creation, not behaviour of the simulationWanted to make it clearer this flag will only affect menu creation, not behaviour of the simulationhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/68443Precision GN2 in HLT2024-02-05T09:36:30+01:00Lucas BezioPrecision GN2 in HLTAddition of a new preicision b-tagger GN220240122 in the HLT code.
Tagging @sfranche @cantel @lshi @lbeemsteAddition of a new preicision b-tagger GN220240122 in the HLT code.
Tagging @sfranche @cantel @lshi @lbeemstehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/68273Adding fastGN2 scores2024-02-09T11:16:00+01:00Lucas BezioAdding fastGN2 scoresAdding fastGN2 inference into BJetFlavourTaggingConfig.py and adding outputs for preselection.
Marked as draft because `PATH_TO_FAST_GN2` in the dev area needs to be confirmed by @lbeemste
Tagging @sfranche & @cantelAdding fastGN2 inference into BJetFlavourTaggingConfig.py and adding outputs for preselection.
Marked as draft because `PATH_TO_FAST_GN2` in the dev area needs to be confirmed by @lbeemste
Tagging @sfranche & @cantelhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/67399TriggerMenuMT: delete obsolete egamma legacy python configuration code2023-11-28T17:42:56+01:00Frank WinklmeierTriggerMenuMT: delete obsolete egamma legacy python configuration codeDelete unused python code.
cc @smh @dbakshigDelete unused python code.
cc @smh @dbakshighttps://gitlab.cern.ch/atlas/athena/-/merge_requests/66006fixing bug in electron triggers for Pb+Pb run2023-09-24T16:52:49+02:00Sebastian Tapia Arayafixing bug in electron triggers for Pb+Pb runProviding the right cells for electron triggers shower shape builder
it was reading unsubtracted cells (pp like) biasing the electrons identification
fixes ATR-28274
tagging HI trigger coordination @asmaga @jkremer, HI conveners @mryba...Providing the right cells for electron triggers shower shape builder
it was reading unsubtracted cells (pp like) biasing the electrons identification
fixes ATR-28274
tagging HI trigger coordination @asmaga @jkremer, HI conveners @mrybar @qhu @angeramihttps://gitlab.cern.ch/atlas/athena/-/merge_requests/64765TrigEDMConfig: use classes for collection properties2023-08-03T17:37:20+02:00Frank WinklmeierTrigEDMConfig: use classes for collection propertiesInstead of using a string as the 4th (optional) collection item in `TriggerEDMRun3`, use a
list of class instances (`InViews`, `Alias`) to configure additional collection properties.
Verified that the list of serialized collections is i...Instead of using a string as the 4th (optional) collection item in `TriggerEDMRun3`, use a
list of class instances (`InViews`, `Alias`) to configure additional collection properties.
Verified that the list of serialized collections is identical before/after this change.
This will make it easier to reuse that 4th item also for other purposes (e.g. truncation mode as discussed in ATR-27986).
cc @tamartin @okuprash @tbold
Merge after https://gitlab.cern.ch/atlas/athena/-/merge_requests/64759.https://gitlab.cern.ch/atlas/athena/-/merge_requests/62640ATR-27398: Fix L1Defined test and correct menu errors2023-04-26T13:07:05+02:00Teng Jian KhooATR-27398: Fix L1Defined test and correct menu errorsAddresses:
- L1Defined test not catching all cases of HLT chains without their L1 items defined in the corresponding L1 menu
- CI errors from HLT menus (lowMu, HI) containing such chains
@fwinkl I still don't know why this wasn't consis...Addresses:
- L1Defined test not catching all cases of HLT chains without their L1 items defined in the corresponding L1 menu
- CI errors from HLT menus (lowMu, HI) containing such chains
@fwinkl I still don't know why this wasn't consistent in CI or locally, but now I just set an empty input file list and trigger L1 json creation when needed. As a bonus now we can correctly check multiseed L1 items.
@dguest, this is so you know I got rid of the janky error parsing. So exceptions in the parsing will now break the script, but before I changed that there were reliably 0 'known errors'. Other options should still be fast + seem to work. L1 check might be marginally slower but works properly.https://gitlab.cern.ch/atlas/athena/-/merge_requests/62359ATR-27270: jet/b-jet trigger physics menu downshift2023-04-19T19:25:20+02:00Dan GuestATR-27270: jet/b-jet trigger physics menu downshiftThis adds new chains for which the jet trigger threshold is moved down by about 6% for almost every lowest unprescaled primary jet and b-jet trigger for `Physics_pp_Run3` menu. This in preparation of a small-R PFlow jet calibration updat...This adds new chains for which the jet trigger threshold is moved down by about 6% for almost every lowest unprescaled primary jet and b-jet trigger for `Physics_pp_Run3` menu. This in preparation of a small-R PFlow jet calibration update at HLT, which improves the HLT jet pT response and resolution but jets incur a ~6% downwards shift in response (HLT jets formerly overcorrected).
Affects trigger counts: Expect higher trigger rate from new chains with current HLT jet calibration.
This was done via a python script by Dan has in [gitlab](https://gitlab.cern.ch/dguest/b-jet-thresholds/-/tree/master): it just adds another otherwise identical menu item with a `#downshift` comment appended. Pinging @khoo, @cantel, @lbeemste, @lshi, @cpollard.
Duplicate of !62353, from a common branch. @lbeemste will have to re-add emojis.https://gitlab.cern.ch/atlas/athena/-/merge_requests/62111Fix un-merged CAs in trigger menu generation (for un-migrated chains)2023-04-06T14:53:09+02:00Francesca PastoreFix un-merged CAs in trigger menu generation (for un-migrated chains)Using @tbold branch that raises error when a un-merged CA is deleted, see https://gitlab.cern.ch/atlas/athena/-/merge_requests/61689, I fixed some failures in the menu code: when chains that are not completely migrated are excluded from ...Using @tbold branch that raises error when a un-merged CA is deleted, see https://gitlab.cern.ch/atlas/athena/-/merge_requests/61689, I fixed some failures in the menu code: when chains that are not completely migrated are excluded from the menu generation, the corresponding CAs are deleted, before being merged. Adding “ca.wasMerged" in various points
Tagging @tbold, @sshawhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/62077Adding anti-bb tagger DL1dbb and gn1+bb test chains2023-04-19T19:25:17+02:00Maggie ChenAdding anti-bb tagger DL1dbb and gn1+bb test chainsA new training of the anti-bb-jet tagger `DL1dbb20230314` trained on mc21 samples for the trigger, and its working points are added to the HLT bjet code. The old version of the tagger `DL1bb20220331` trained on mc16 samples have been rem...A new training of the anti-bb-jet tagger `DL1dbb20230314` trained on mc21 samples for the trigger, and its working points are added to the HLT bjet code. The old version of the tagger `DL1bb20220331` trained on mc16 samples have been removed.
3 new test chains containing DL1dbb have been added to the Run 3 dev menu:
`HLT_j80c_020jvt_j55c_020jvt_j28c_020jvt_j20c_020jvt_SHARED_2j20c_020jvt_bgn182bb96_pf_ftf_presel2c20XX2c20b85_L1J45p0ETA21_3J15p0ETA25`
`HLT_j80c_020jvt_j55c_020jvt_j28c_020jvt_j20c_020jvt_SHARED_2j20c_020jvt_bgn177bb96_pf_ftf_presel2c20XX2c20b85_L1J45p0ETA21_3J15p0ETA25`
`HLT_j80c_020jvt_j55c_020jvt_j28c_020jvt_j20c_020jvt_SHARED_2j20c_020jvt_bgn175bb90_pf_ftf_presel2c20XX2c20b85_L1J45p0ETA21_3J15p0ETA25`
The old chains containing DL1d and mc16 version of DL1dbb have been removed.
The following tests have been successfully ran:
- 3 out of the 4 `runTrigART.py -m -j4` (`test_trigP1_v1Dev_decodeBS_build.py` failed at `CountRefComp` since new chains have been added, and no references to be generated and compared to)
- `ctest -j4`
Note: this should close ATR-26380 (automatically, via gitlab magic!)Maggie ChenMaggie Chenhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/61187Migrated couning algs in MinBias to CA2023-03-06T09:42:49+01:00Tomasz BoldMigrated couning algs in MinBias to CAAs the title says, some more algs are now CA configure with this MR. tested with: test_trig_data_v1lowMu_build.py
Config diff:
```
Run with arguments:
confTool.py --diff oldcfg.pkl cfg.pkl
Step 1: reference file #components: 14484
Step 2...As the title says, some more algs are now CA configure with this MR. tested with: test_trig_data_v1lowMu_build.py
Config diff:
```
Run with arguments:
confTool.py --diff oldcfg.pkl cfg.pkl
Step 1: reference file #components: 14484
Step 2: file to check #components: 14484
Legend:
Differences in components Settings in 1st file Settings in 2nd file
Component PerfMonMTSvc may differ
wallTimeOffset = 1677740000000.0 vs 1677750000000.0 <<
but all are suppressed by renaming/known differences/...
```
Histograms diff:
```
rootcomp.py old-expert-monitoring.root expert-monitoring.root
----------------------------------------------------------------------
Command : rootcomp.py old-expert-monitoring.root expert-monitoring.root
Reference : old-expert-monitoring.root
File : expert-monitoring.root
Comparison : BIN-BY-BIN (1.0e-06)
Ignored histograms: Unpck$, BufFreeCnt$, CalEvtSize$, /TIME_, /athenaHLT.*/.*Time$, HltEventLoopMgr/.*Time.*, HltEventLoopMgr/PopScheduler.*, MessageSvc/MessageCount, TrigSignatureMoni/.*Rate, TrigOpMonitor/GeneralOpInfo, TrigOpMonitor/IOVDb.*, TrigOpMonitor/.*
ReadTime, TrigOpMonitor/.*BytesRead, HLTFramework/ROBDataProviderSvc, HLTFramework/SchedulerMonSvc, HLTSeeding/Random
----------------------------------------------------------------------
Summary:
========
Total histograms compared: 2032
Missing references: 0
Matching histograms: 2032
Different histograms: 0
Ignored histograms: 152
```
FI: @kcieslahttps://gitlab.cern.ch/atlas/athena/-/merge_requests/60253TriggerMenuMT: pass flags to MenuSequence[CA]2023-02-02T21:44:10+01:00Frank WinklmeierTriggerMenuMT: pass flags to MenuSequence[CA]Explicitly pass the ConfigFlags to `MenuSequence` and `MenuSequenceCA`.
Only the latter would be strictly required but the former is useful to
make progress in eliminating the use of the global `ConfigFlags`.
Relates to ATR-24994 and re...Explicitly pass the ConfigFlags to `MenuSequence` and `MenuSequenceCA`.
Only the latter would be strictly required but the former is useful to
make progress in eliminating the use of the global `ConfigFlags`.
Relates to ATR-24994 and requires https://gitlab.cern.ch/atlas/athena/-/merge_requests/60252.
cc @fpastorehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/59221FTag training run 4 in derivation2023-08-21T16:13:29+02:00Thomas StreblerFTag training run 4 in derivationThis MR:
- renames `BTagRun3Config.py` into `BTagConfig.py`
- renames `FtagRun3DerivationConfig.py` into `FtagDerivationConfig.py`
- forward the CA config flags through the `SlimmingHelper` to be used to configure FTAG content + update a...This MR:
- renames `BTagRun3Config.py` into `BTagConfig.py`
- renames `FtagRun3DerivationConfig.py` into `FtagDerivationConfig.py`
- forward the CA config flags through the `SlimmingHelper` to be used to configure FTAG content + update accordingly the CA configs for all derivation formats available
- adds configuration to run Run 4 FTAG trainings + store associated variables in derivation
FYI @duperrin @bdong @ligang @sgoswami @khanovhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/51419Cleanup btagging configuration2022-04-05T20:58:05+02:00Dan GuestCleanup btagging configurationThis consolidates the configuration functions we use in several places:
- [trigger](https://gitlab.cern.ch/atlas/athena/-/blob/master/Trigger/TriggerCommon/TriggerMenuMT/python/HLT/Bjet/BjetFlavourTaggingConfiguration.py),
- [reconstruc...This consolidates the configuration functions we use in several places:
- [trigger](https://gitlab.cern.ch/atlas/athena/-/blob/master/Trigger/TriggerCommon/TriggerMenuMT/python/HLT/Bjet/BjetFlavourTaggingConfiguration.py),
- [reconstruction](https://gitlab.cern.ch/atlas/athena/-/blob/master/PhysicsAnalysis/JetTagging/JetTagAlgs/BTagging/python/BTagRun3Config.py),
- [derivations](https://gitlab.cern.ch/atlas/athena/-/blob/master/PhysicsAnalysis/DerivationFramework/DerivationFrameworkFlavourTag/python/FtagRun3DerivationConfig.py), and
- [retagging](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/-/blob/r22/BTagTrainingPreprocessing/python/retag.py) (which doesn't happen in the Athena repo).
Overview
--------
Pictures are cool:
```mermaid
graph TD;
subgraph BTagAlgs [Configured by One Function]
assoc[JetParticleAssociation] --> jettag & finder
assocm[JetParticleAssociation muons] --> jettag
finder[JetSecVtxFinding] --> vx[JetSecVertexing]
vx --> jettag
jettag[JetBTagging] ==> muaug[BTagMuonAugmenter] & jetaug[BTagJetAugmenter]
muaug & jetaug ==> dl[Machine Learning Algorithms]
end
db[Calibration Database] --> jettag
tr(tracks) -.-> aug & vx & jetaug & dl
mu(Muons) -.-> assocm & muaug
aug[BTagTrackAugmenter] --> assoc
jet(jets) -.-> assoc & assocm & finder & vx & jettag
pv(Primary Vertex) -.-> finder & vx & jettag
json(List of NN Files) -.-> dl
dl ==> btag(BTagging Object)
```
This is a rough sketch of how information flows through the b-tagging code. Little boxes are algorithms, and the dotted lines indicate where some information needs to be passed into them. Solid lines indicate objects that are created internally. In the solid line case, the name of the object has to be synchronized between algorithms. The `BTagging` object follows the thick black line.
The key point here is that **this logic is implemented in 4 different places.**
The problem
-----------
Modular stuff is good. But in this case the 4 implementations are all supposed to do the same thing. The underlying code is also a bit crufty and some of the boxes should probably be split or merged, which is really hard to do in a few places coherently.
So for now it's probably better to merge everything. The big gray box holds the algorithms we configure with one function. **We want to put everything in the gray box!** Once we've done that we can move stuff around to make it easier to pop new boxes in and take old boxes out.
Implementation
--------------
The idea was to move everything into the box, but that turned out to be a bit more difficult. Instead most of the tagging calls three functions:
1. a calibration database setup,
1. track augmentation, and
1. the top level b-tagging one (the gray box).
I wasn't able to merge the first two into the last because:
- The conditions database setup function for derivations can't be replaced by `JetTagCalibCfg` for some reason. It seems to break the muon conditions alg when I try.
- Track augmentation has to be separated from the rest of tagging, because the retagging code uses a view container which is the union of two other containers. Both of these have to be augmented _before_ the containers are merged.
In the process of implementing this I made some other improvements:
- ATLASRECTS-6635: Some small progress on cleaning up `CompFactory` calls on imports
- ATLASRECTS-6172: Add soft muon scalars to the `BTagging` object
- Move configuration functions that call `FlavorTagDiscriminants` into `FlavorTagDiscriminants`
- Add or clean up some `ConfigFlags`:
- Merged `run2TaggersList` and `Run2TrigTaggers` into `taggerList`. They are the same taggers, but if they ever diverge the `ConfigFlags` aren't shared between reconstruction, trigger, and derivations anyway.
- Made the `taggerList` depend on whether we've enabled `RunFlipTaggers`
- Moved `calibrationChannelAliases` to `ConfigFlags`, cleaned it up considerably
- Added a `forcedCalibrationChannel` option, which tells every tagger to use a specific calibration channel
- Updated the "retagging" store gate renaming functions to be the ones we actually use for retagging
I also deleted and simplified a lot of unused code.
Validation
----------
This causes no changes in any physics outputs (I checked trigger and derivations). I've done some tests on `DAOD_PHYS` and `DAOD_FTAG1`. The only changes to FTAG1 (over a few hundred events) were the addition of `softMuon` variables.
Built on nightly `2022-03-21T2101`
Implications for developers
---------------------------
A few things have moved around, so I'll give a short guide on where to find them now. Everything that runs the main tagging chain will now have a call like
```python
BTagAlgsCfg(cfgFlags, JetCollection, nnList)
```
Which is defined in [BTagging/BTagRun3Config.py](https://gitlab.cern.ch/atlas/athena/-/blob/master/PhysicsAnalysis/JetTagging/JetTagAlgs/BTagging/python/BTagRun3Config.py). The `nnList` is a list of all dips and dl1 taggers for that specific collection. There are also optional arguments for the `trackCollection`, `muons`, and `primaryVertices` (by default they are the standard offline ones).
A lot more options have also been moved to [BTagging/BTaggingConfigFlags.py](https://gitlab.cern.ch/atlas/athena/-/blob/master/PhysicsAnalysis/JetTagging/JetTagAlgs/BTagging/python/BTaggingConfigFlags.py). These include `calibrationChannelAliases` and the `taggerList`, the later of which has a default that depends on whether the flip taggers are enabled.
To Do
-----
I left out a few things that should be discussed with the flavor tagging group, or that might depend on external developments:
- Figure out why I can't use the same calibration database setup function in derivations.
- Do track collection merging as part of this function.
- Consider using `forcedCalibrationChannel` in more places. We might have PFlow specific trainings for taggers that use the calibration database, and I'm pretty sure we don't have anything specific for variable radius track jets. If the trainings are all identical we could replace the channel aliases with an empty list and map everything to one jet collection.
- Enable muon information in reconstruction jobs. Right now the data dependencies for BTagMuonAugmenter aren't correct, which leads to random crashes in Athena MT. Derivations are single thread for now, so the muons still run there.
- Disable `MV2c10` in the trigger code. See ATR-25239.https://gitlab.cern.ch/atlas/athena/-/merge_requests/50489update of mbts items/chains2022-02-16T15:44:25+01:00Sarka Todorovaupdate of mbts items/chainsUpdate of L1 MBTS items in (HI) L1 menu.
Update of HLT MBTS chains (+supporting trigger).
Streaming to be reviewed.
Jira tickets: ATR-24701 , ATR-24717Update of L1 MBTS items in (HI) L1 menu.
Update of HLT MBTS chains (+supporting trigger).
Streaming to be reviewed.
Jira tickets: ATR-24701 , ATR-24717Sarka TodorovaSarka Todorovahttps://gitlab.cern.ch/atlas/athena/-/merge_requests/45430Refactored functions for memoization2021-09-15T12:04:27+02:00Kacper Wojciech TopolnickiRefactored functions for memoizationFunctions from -generateTau.py- were written as
a combination of two functions, one of which can
be momoized using the -AccumulatorCache- decorator.
@tbold @rbielski ATR-23200Functions from -generateTau.py- were written as
a combination of two functions, one of which can
be momoized using the -AccumulatorCache- decorator.
@tbold @rbielski ATR-23200