From 3b4b1f1af72e4fb8121f7bf1ecf2ffc3cefaf9cd Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Tue, 20 May 2025 16:34:40 +0200
Subject: [PATCH 1/9] Fix link in trigger developer algos.md

---
 docs/athena/trigger/developers/algos.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/athena/trigger/developers/algos.md b/docs/athena/trigger/developers/algos.md
index ff9e7d9..d0b651c 100644
--- a/docs/athena/trigger/developers/algos.md
+++ b/docs/athena/trigger/developers/algos.md
@@ -49,7 +49,7 @@ need to be recorded in the output file these steps should be followed:
 
 1. contact trigger EDM coordinator [found on trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware)
 1. in the python configuration of the algorithm an instance of the
-  `recordable` function from [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun.py#L34) needs
+  `recordable` function from [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py#L34) needs
   to be used when setting the name of the collection. It assures that
   the naming convention of the HLT EDM is followed and no trivial typos sneak in between name of the container .
 1. The collection (type and name)  has to be listed in 
-- 
GitLab


From b1c29a85fe8b4f4b5e3c71b9779ebc56d613f088 Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Tue, 20 May 2025 16:35:54 +0200
Subject: [PATCH 2/9] WIP: Add trigger developers dedicated docs on trigger
 EDM.

---
 docs/athena/trigger/developers/edm.md | 79 +++++++++++++++++++++++++++
 mkdocs.yml                            |  1 +
 2 files changed, 80 insertions(+)
 create mode 100644 docs/athena/trigger/developers/edm.md

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
new file mode 100644
index 0000000..c1e3dc1
--- /dev/null
+++ b/docs/athena/trigger/developers/edm.md
@@ -0,0 +1,79 @@
+# Trigger Event Data Model
+
+The Trigger Event Data Model (EDM) defines the trigger reconstructed content that is written out to RAW files and data or MC AODs.
+The Trigger EDM content for Run 3 is defined in [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py) (the same folder contains similar files for other periods).
+Each EDM entry is a tuple containing all EDM details for a specific interface or Aux container. An example of two (interface + aux) EDM entries:
+```python
+('xAOD::CaloClusterContainer#HLT_CaloEMClusters_Electron',               'BS ESD AODFULL AODSLIM', 'Egamma', [InViews('precisionCaloElectronViews')]),
+('xAOD::CaloClusterTrigAuxContainer#HLT_CaloEMClusters_ElectronAux.',    'BS ESD AODFULL AODSLIM', 'Egamma'),
+
+('xAOD::CaloClusterContainer#HLT_TopoCaloClustersLC',                             'BS ESD AODFULL', 'Tau', [InViews('tauCaloMVAViews'), allowTruncation]),   
+('xAOD::CaloClusterTrigAuxContainer#HLT_TopoCaloClustersLCAux.nCells.CENTER_MAG', 'BS ESD AODFULL', 'Tau', [allowTruncation]),
+
+```
+Every EDM entry will at minimum define the container type and container name, the "EDM targets" and trigger signature. Any additional dynamic Aux variables
+to be recorded are specified as part of the Aux container name string.
+The EDM targets define the output type to which the containers will be recorded. We use these to restrict the HLT content when not all HLT content is needed,
+e.g. in MC samples for analysis. The trigger signature is used to categorise total EDM size by signature in the Trigger EDM monitoring. (FIXME REF)
+
+## EDM targets
+
+The EDM target labels (2nd entry in an EDM entry) are mapped to the following output:
+
+| Label | Output | Purpose |
+|-------------|-------------|-------------|
+|`BS`| ByteStream - Any content that is to make it out of Point 1 and be recorded to RAW file. | Required for recording to data AOD after or kept in BS for debugging in case of data taking issues.|
+|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies. |
+|`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning. |
+|`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors. |
+|`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information. |
+
+**Note**: SampleA trigger validation productions record all `ESD` labelled content.
+
+:alert:
+**Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
+* Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
+* Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+
+
+## Optional EDM details
+
+There are several additional EDM details that can be specified as a list constituting the last entry in the tuple.
+
+### Views
+
+This names the event view in which this collection is reconstructed in, in the case of RoI reconstruction. For more details, see [the section on EventViews](eventviews.md).
+
+### allowTruncation
+
+This is only valid for data taking. It is an object to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, ""allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
+
+### Aliases
+
+A name with suffix `ShallowCopy` is set for all `ShallowAuxContainer` type containers to indicate that they need special treatment by the [HLTEDMCreator]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TrigSteer/TrigOutputHandling/src/HLTEDMCreator.h) given that the Aux store name differs from the interface name.
+
+### EDM manipulation
+
+The following configuration flag setting will override the "output" type for a job:
+```python
+flags.Trigger.AODEDMSet='ESD'
+```
+This will record all `ESD` labelled containers, even if the output is an AOD.
+
+
+The following configuration flag settings will override or add a new EDM entry if not already found:
+```python
+flags.Trigger.ExtraEDMList = [
+    ('xAOD::CaloClusterContainer#HLT_TopoCaloClustersFS',                  'BS ESD AODFULL', 'Calo'),
+    ('xAOD::CaloClusterTrigAuxContainer#HLT_TopoCaloClustersFSAux.nCells', 'BS ESD AODFULL', 'Calo'),
+    ('xAOD::ElectronContainer#HLT_NewParticles',                                'BS ESD AODFULL', 'EGamma', [InViews('NewViews')]),
+    ('xAOD::ElectronAuxContainer#HLT_NewParticlesAux.',                'BS ESD AODFULL', 'EGamma'),
+]
+```
+
+This command will update the targets for the already existing `HLT_TopoCaloClustersFS` and `HLT_TopoCaloClustersFSAux` containers, as well as add new `HLT_NewParticles` and `HLT_NewParticlesAux` containers.
+
+
+## unit tests
+
+## Procedure for adding/removing trigger EDM content
diff --git a/mkdocs.yml b/mkdocs.yml
index be30c47..bb4353d 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -149,6 +149,7 @@ nav:
       - athena/trigger/developers/hypos.md
       - athena/trigger/developers/combohypos.md
       - athena/trigger/developers/navigation.md
+      - athena/trigger/developers/edm.md
       - athena/trigger/developers/monitoring.md
       - athena/trigger/developers/menu.md
       - athena/trigger/developers/validation.md
-- 
GitLab


From e0e0776d1fc5bbf23d583be647ae8f581f31b39a Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Wed, 21 May 2025 16:54:34 +0200
Subject: [PATCH 3/9] WIP: Further additions to trigger EDM docs, removing some
 from trigger reco algos docs

---
 docs/athena/trigger/developers/algos.md |  8 ++--
 docs/athena/trigger/developers/edm.md   | 51 ++++++++++++++++++++++---
 2 files changed, 49 insertions(+), 10 deletions(-)

diff --git a/docs/athena/trigger/developers/algos.md b/docs/athena/trigger/developers/algos.md
index d0b651c..a6cbd7d 100644
--- a/docs/athena/trigger/developers/algos.md
+++ b/docs/athena/trigger/developers/algos.md
@@ -44,18 +44,16 @@ ready) [fast e/gamma cluster maker algorithm]({{data.athena_git_url}}/blob/{{dat
 
 
 ## Saving EDM objects
-If the reconstructed objects that are created by algorithms
-need to be recorded in the output file these steps should be followed:
+The following software modifications are needed to record reconstructed objects created by HLT algorithms to file:
 
-1. contact trigger EDM coordinator [found on trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware)
 1. in the python configuration of the algorithm an instance of the
   `recordable` function from [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py#L34) needs
   to be used when setting the name of the collection. It assures that
   the naming convention of the HLT EDM is followed and no trivial typos sneak in between name of the container .
-1. The collection (type and name)  has to be listed in 
+1. The collection (type and name)  has to be listed in
   [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py)
   together with the required output file types in which it should be recorded. Typically the object should be stored
-  at least in `BS`, `ESD` and `AOD`. Consult the Trigger EDM coordinator about this. Especially about the `AOD` formats where the size is constrained.
+  at least in `BS` and `ESD`.
 1. Ensure that in
   [HLTEDMCreator.h]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TrigSteer/TrigOutputHandling/src/HLTEDMCreator.h)
   and
diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index c1e3dc1..85f1548 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -1,4 +1,4 @@
-# Trigger Event Data Model
+# Trigger EDM
 
 The Trigger Event Data Model (EDM) defines the trigger reconstructed content that is written out to RAW files and data or MC AODs.
 The Trigger EDM content for Run 3 is defined in [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py) (the same folder contains similar files for other periods).
@@ -32,8 +32,8 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
 
 :alert:
 **Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
-* Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
-* Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
+- Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
 
 
 ## Optional EDM details
@@ -52,7 +52,7 @@ This is only valid for data taking. It is an object to indicate that this result
 
 A name with suffix `ShallowCopy` is set for all `ShallowAuxContainer` type containers to indicate that they need special treatment by the [HLTEDMCreator]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TrigSteer/TrigOutputHandling/src/HLTEDMCreator.h) given that the Aux store name differs from the interface name.
 
-### EDM manipulation
+## EDM manipulation
 
 The following configuration flag setting will override the "output" type for a job:
 ```python
@@ -76,4 +76,45 @@ This command will update the targets for the already existing `HLT_TopoCaloClust
 
 ## unit tests
 
-## Procedure for adding/removing trigger EDM content
+Two unit tests exist that check for:
+
+- Validity of EDM entries.
+- Ordering of EDM entries.
+
+
+## Procedure for modifying trigger EDM
+
+### Adding content
+
+The following steps should be followed if you would like to **add** new trigger EDM content:
+
+1. Do the necessary software modications to record the new content as described in [section Reconstruction Algorithms: Saving EDM objects](algos.md#saving-edm-objects).
+1. Test and validate: Check content is recorded in the expected output (data AOD, MC AOD etc) and the content is correct. See the subsection below!
+1. Estimate the size increase. See the subsection below!
+1. Contact the trigger EDM coordinator [found on trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
+    (a) a JIRA ticket motivating the new implementation, new EDM content and EDM target, as well as a post of the results of the 2 previous steps.
+    (b) (if ready) a link to the draft Merge Request.
+1. The trigger EDM coordinator will confirm whether the EDM targets are appropriate and sufficiently motivated.
+1. If the change is implemented during a data taking period (e.g. Run 3) to the current Tier0 branch ({{data.branchTier0Short}}), the change requires "PROC" sign-off (Prompt Reconstruction Operation Coordinator) due to the _Frozen Tier0 Policy_ which tries to prevent EDM changes during data taking periods or major MC campaigns -- see details on the [FrozenTier0 twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy).
+If relevant, you should tag or email PROC after sign-off from the trigger group (FIXME: Check what actual procedure should be). You can find the current coordinators under the [ATLAS Data Preparation organisation page](https://atlaspo.cern.ch/public/ATLASOrganisation/index.html?value=DATAPREP) or on the twiki.
+1. Once the MR is undrafted, there are a couple of CI tests that may fail due to breaking the Frozen Tier0 policy during data taking, namely `CITest_RecoRun3Data_Checks-test` and `CITest_RecoRun3MC-test`. The ERROR will include
+```sh
+ERROR    Py:diff-root        ERROR files differ!
+ERROR    Your change breaks the frozen tier0 policy in test q449.
+ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
+```
+along with details of the diff. The failure is due to outdated AOD/ESD references, which PROC manages. Double-check that the differences are expected and tag the current PROC in your MR.
+The MR will only be merged once the ref updates have been done by PROC (once all tests are green). (TODO: Check - twiki says developers can update references themselves.)
+
+**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator also needs to, post-MR, update the bytestream schema, so that new container types can be serialised as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
+
+**Advice** Usually a new EDM implementation comes along with new trigger chains which need to be motivated in a presentation at the Trigger General Meeting if they are ready to go into the Physics menu. This presentation should also include material motivating the EDM content and the expected EDM size increase.
+
+### Removing content
+
+If you would like to **remove** trigger EDM content, then get in touch with the [trigger EDM coordinator](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware) directly, so they can make sure that the content is not used by anyone. **Note** A trigger EDM entry should never be completely removed; the EDM targets should be removed instead.
+
+### Test and validate
+
+
+### Estimate size increase
-- 
GitLab


From fa4403988c02c5eb18529d7a6b39cafbb4828234 Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Thu, 22 May 2025 18:35:24 +0200
Subject: [PATCH 4/9] WIP: Adding test and validation to trigger EDM docs,
 fixes and todos

---
 docs/athena/trigger/developers/edm.md | 80 ++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 3 deletions(-)

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index 85f1548..fabdedc 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -30,8 +30,8 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
 
 **Note**: SampleA trigger validation productions record all `ESD` labelled content.
 
-:alert:
 **Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
+
 - Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
 - Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
 
@@ -42,7 +42,7 @@ There are several additional EDM details that can be specified as a list constit
 
 ### Views
 
-This names the event view in which this collection is reconstructed in, in the case of RoI reconstruction. For more details, see [the section on EventViews](eventviews.md).
+This names the event view in which this collection is reconstructed in, in the case of reconstruction in a limited region of the detector, so that additionally information allowing the items to be accessed in an event view, is recorded. For more details, see [the section on EventViews](eventviews.md).
 
 ### allowTruncation
 
@@ -76,6 +76,8 @@ This command will update the targets for the already existing `HLT_TopoCaloClust
 
 ## unit tests
 
+TODO!
+
 Two unit tests exist that check for:
 
 - Validity of EDM entries.
@@ -114,7 +116,79 @@ The MR will only be merged once the ref updates have been done by PROC (once all
 
 If you would like to **remove** trigger EDM content, then get in touch with the [trigger EDM coordinator](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware) directly, so they can make sure that the content is not used by anyone. **Note** A trigger EDM entry should never be completely removed; the EDM targets should be removed instead.
 
-### Test and validate
+## Test and validate
+
+The following should be cross-checked for new EDM content:
+
+1. New EDM containers appear in RAW or relevant AOD output.
+1. New EDM containers are populated for events that seed the relevant reconstruction (this is not only events where the relevant chain has passed).
+1. Dynamic values can be accessed and make sense: Are not default values, zeroed, etc.
+1. The size increase is as expected and not exhorbitant.
+
+The results should be documented in JIRA.
+
+!!! warning
+    All EDM containers listed in `TriggerEDMRunX` _will_ appear in the relevant output specified by the EDM target, _regardless_ if they were reconstructed at the HLT: We run a "GapFiller" each time we process produce AODs, which creates empty HLT containers for an event if they don't already exist to ensure a consistent EDM throughout all events. Thus, seeing a container listed doesn't mean it's populated!
+
+### Cross-check: Container recording
+
+The following tests will output AODs after rerunning the HLT on a given menu:
+
+| test | data/MC | EDM target | Menu |
+|-------------|-------------|-------------|-------------|
+| test_trigAna_RDOtoAOD_v1MC_build | ttbar MC | AODSLIM | `"Physics_pp_run3"` |
+| test_trigAna_RDOtoAOD_v1Dev_build | ttbar MC | AODFULL | `"Dev_pp_run3"`|
+| test_trigP1_v1PhysP1_T0Mon_build | data | AODFULL | `"Physics_pp_run3"`|
+
+Each test will output `AOD.pool.root.checkFile` and `AOD.pool.root.checkFileTrigSize` files, produced from
+`checkFile.py AOD.pool.root` and `checkFileTrigSize.py AOD.pool.root`, respectively.
+
+Your container should appear in `AOD.pool.root.checkFile`. Each file provides details on container or total component sizes.
+
+#### Tips on running own trigger tests
+
+When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly or check the log file for the executed command. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
+
+For example, using an existing trigger test command file/log output as a reference, commands may look something like listed below.
+
+- To create an AOD on MC ttbar, `"Dev_pp_run3"` menu and `AODSLIM` target:
+```sh
+Reco_tf.py --outputAODFile=AOD.pool.root --steering "doRDO_TRIG" --CA "all:True"  --preExec "RDOtoRDOTrigger:flags.Exec.FPE=1" "all:flags.Trigger.triggerMenuSetup='Dev_pp_run3_v1';flags.Trigger.AODEDMSet='AODSLIM';" --preInclude "all:Campaigns.MC23e" --conditionsTag "default:OFLCOND-MC23-SDR-RUN3-08" --athenaopts="--imf --threads=1" --maxEvents=20 --inputRDOFile=root://eosatlas.cern.ch//eos/atlas/atlascerngroupdisk/data-art/large-input/trig-val/TriggerTest/valid1.601229.PhPy8EG_A14_ttbar_hdamp258p75_SingleLep.recon.RDO.e8514_e8528_s4369_s4370_r16083_tid42189392_00/RDO.42189392._000001.pool.root.1,root://eosatlas.cern.ch//eos/atlas/atlascerngroupdisk/data-art/large-input/trig-val/TriggerTest/valid1.601229.PhPy8EG_A14_ttbar_hdamp258p75_SingleLep.recon.RDO.e8514_e8528_s4369_s4370_r16083_tid42189392_00/RDO.42189392._000002.pool.root.1 >Reco_tf.log 2>&1
+```
+
+- To create an AOD on data, `"Dev_pp_run3"` menu and `AODFULL` target:
+```sh
+Trig_reco_tf.py --CA  --preExec "BSRDOtoRAW:Exec.FPE=1" "Trigger.triggerMenuSetup='Dev_pp_run3_v1_HLTReprocessing_prescale' Trigger.doLVL1=True" --prodSysBSRDO True --outputBSFile=RAW.pool.root --outputHIST_HLTMONFile=hltmon.root --outputDRAW_TRIGCOSTFile=TRIGCOST.pool.root --outputNTUP_TRIGCOSTFile=cost.ntup.root --runNumber 475321 --athenaopts="--imf --threads=4 --concurrent-events=4 --nprocs=1" --maxEvents=50 --inputBS_RDOFile=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-11._0001.1,/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-12._0001.1 >Trig_reco_tf.BSRDOtoRAW.log 2>&1
+
+Trig_reco_tf.py --inputBSFile=RAW.pool.root --outputAODFile=AOD.pool.root --outputNTUP_TRIGRATEFile=rate.ntup.root --outputHISTFile=ExampleMonitorOutput.root --conditionsTag="<conditions-tag>" --geometryVersion='<geometry-version>' --preExec="flags.Trigger.triggerMenuSetup='Dev_pp_run3_v1_HLTReprocessing_prescale'; flags.Trigger.AODEDMSet='AODFULL'; flags.DQ.Steering.doDataFlowMon=True; flags.DQ.Steering.doHLTMon=True; flags.DQ.Steering.doLVL1CaloMon=True; flags.DQ.Steering.doGlobalMon=True; flags.DQ.Steering.doLVL1InterfacesMon=True; flags.DQ.Steering.doCTPMon=True; flags.DQ.Steering.HLT.doBjet=True; flags.DQ.Steering.HLT.doInDet=True; flags.DQ.Steering.HLT.doBphys=True; flags.DQ.Steering.HLT.doCalo=True; flags.DQ.Steering.HLT.doEgamma=True; flags.DQ.Steering.HLT.doJet=True; flags.DQ.Steering.HLT.doMET=True; flags.DQ.Steering.HLT.doMinBias=True; flags.DQ.Steering.HLT.doMuon=True; flags.DQ.Steering.HLT.doTau=True;" --CA --athenaopts="--imf --threads=4 --concurrent-events=4" --maxEvents=50 >Trig_reco_tf.Tier0Reco.log 2>&1
+
+checkFile.py AOD.pool.root >AOD.pool.root.checkFile 2>&1;
+checkFileTrigSize.py AOD.pool.root >AOD.pool.root.checkFileTrigSize 2>&1
+```
+
+### Cross-check: Container content
+
+First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag, e.g.:
 
+```sh
+athenaHLT.py -o output --imf --perfmon --threads=4 --concurrent-events=4 --nprocs=1 --number-of-events=50 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-11._0001.1 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-12._0001.1 TriggerJobOpts.runHLT Trigger.triggerMenuSetup="PhysicsP1_pp_run3_v1_HLTReprocessing_prescale" Trigger.selectChains=[\"HLT_j250_a10sd_cssk_ditauOmni0Trk99_pf_jes_ftf_preselj200_L1jJ160\",\"HLT_tau200_mediumGNTau_L1eTAU140\"] Trigger.doLVL1=True Exec.FPE=1 >athenaHLT.log 2>&1
+```
+TODO: This is for data. Easiest way to get skimmed MC on limited menu?
+
+This ensures that your output contains only events that passed your chain, meaning containers should be populated
+and ensures that all reconstruction is performed by _your_ chains
+(relevant if you are adding new chains that use already existing reconstructed objects).
+
+Use an analysis script to retrieve the containers and plot variables to histogram or print to terminal to check that variables look correct.
 
 ### Estimate size increase
+
+This can be tricky, especially if your chains have a low selection rate. A size estimate can be given by comparing between the latest Athena branch and your implementation:
+
+- Size of produced RAW (data) files between the current release and your implementation.
+- AOD file size difference quoted in AOD.pool.root.checkFile or summed size of new containers.
+- Trig EDM size difference quoted in AOD.pool.root.checkFileTrigSize.
+
+TODO: Ran out of time. Needs more detail.
+
+Once the implementation is merged to Athena, ART (nightly) tests monitor the trigger EDM size on larger amount (~1000) of MC & data events. These numbers can be referenced in the JIRA. See [the section on trigger EDM size monitoring](validation.md#trigger-edm-size-monitoring) on obtaining the size estimates.
-- 
GitLab


From f19ff50ac84c8bb2f3a1e33a2587f48f5a8babc1 Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Fri, 30 May 2025 15:56:33 +0200
Subject: [PATCH 5/9] address comments for trigger edm section

---
 docs/athena/trigger/developers/algos.md | 10 ++++---
 docs/athena/trigger/developers/edm.md   | 39 +++++++------------------
 2 files changed, 17 insertions(+), 32 deletions(-)

diff --git a/docs/athena/trigger/developers/algos.md b/docs/athena/trigger/developers/algos.md
index a6cbd7d..c9a7df4 100644
--- a/docs/athena/trigger/developers/algos.md
+++ b/docs/athena/trigger/developers/algos.md
@@ -47,13 +47,13 @@ ready) [fast e/gamma cluster maker algorithm]({{data.athena_git_url}}/blob/{{dat
 The following software modifications are needed to record reconstructed objects created by HLT algorithms to file:
 
 1. in the python configuration of the algorithm an instance of the
-  `recordable` function from [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py#L34) needs
+  `recordable` function from [TrigEDMConfig.TriggerEDM]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDM.py#L34) needs
   to be used when setting the name of the collection. It assures that
   the naming convention of the HLT EDM is followed and no trivial typos sneak in between name of the container .
 1. The collection (type and name)  has to be listed in
-  [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py)
+  [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py) (or `TriggerEDMRun4` for upgrade studies)
   together with the required output file types in which it should be recorded. Typically the object should be stored
-  at least in `BS` and `ESD`.
+  at least in `BS` and `ESD` (`RDO_TRIG` files - the equivalent of "raw" files for MC - contain all `ESD`-labelled content).
 1. Ensure that in
   [HLTEDMCreator.h]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TrigSteer/TrigOutputHandling/src/HLTEDMCreator.h)
   and
@@ -67,7 +67,9 @@ The following software modifications are needed to record reconstructed objects
 Once the above steps are done the object should appear in the relevant output files.
 For POOL files (ESD, AOD) the content can be checked with one of the following scripts:
 `checkxAOD.py filename` `checkSG.py filename` `checkFile.py filename`.
-  
+
+See the [trigger EDM guidelines](edm.md#procedure-for-modifying-trigger-edm) on the full procedure for adding new trigger EDM content (testing, verification and approval).
+
 
 ## Troubleshooting
 Typically a reconstruction algorithm uses many input data
diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index fabdedc..05b978a 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -13,8 +13,8 @@ Each EDM entry is a tuple containing all EDM details for a specific interface or
 ```
 Every EDM entry will at minimum define the container type and container name, the "EDM targets" and trigger signature. Any additional dynamic Aux variables
 to be recorded are specified as part of the Aux container name string.
-The EDM targets define the output type to which the containers will be recorded. We use these to restrict the HLT content when not all HLT content is needed,
-e.g. in MC samples for analysis. The trigger signature is used to categorise total EDM size by signature in the Trigger EDM monitoring. (FIXME REF)
+The EDM targets define the output type to which the containers will be recorded in the case of `RAW`, `RDO_TRIG`, `ESD` and `AOD` files, as well as PHYSVAL `DAOD` files. The `ESD` level is used for MC `RDO_TRIG` (equivalent to data `RAW` files for MC) production. We use these to restrict the HLT content when not all HLT content is needed,
+e.g. in MC AODs for physics analysis derivations. The trigger signature is used to categorise total EDM size by signature in the [Trigger EDM monitoring](validation.md#trigger-edm-size-monitoring).
 
 ## EDM targets
 
@@ -23,16 +23,16 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
 | Label | Output | Purpose |
 |-------------|-------------|-------------|
 |`BS`| ByteStream - Any content that is to make it out of Point 1 and be recorded to RAW file. | Required for recording to data AOD after or kept in BS for debugging in case of data taking issues.|
-|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies. |
-|`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning. |
-|`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors. |
-|`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information. |
+|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies, also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
+|`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning.|
+|`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors.|
+|`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information.|
 
 **Note**: SampleA trigger validation productions record all `ESD` labelled content.
 
 **Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
 
-- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
+- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
 - Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
 
 
@@ -42,11 +42,11 @@ There are several additional EDM details that can be specified as a list constit
 
 ### Views
 
-This names the event view in which this collection is reconstructed in, in the case of reconstruction in a limited region of the detector, so that additionally information allowing the items to be accessed in an event view, is recorded. For more details, see [the section on EventViews](eventviews.md).
+In the case of reconstruction in a limited region of the detector, this names the event view in which this collection is reconstructed, so that additional information allowing the items to be accessed in event views is recorded. For more details, see [the section on Event Views](eventviews.md).
 
 ### allowTruncation
 
-This is only valid for data taking. It is an object to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, ""allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
+This is only valid for data taking. It is a flag to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
 
 ### Aliases
 
@@ -54,11 +54,11 @@ A name with suffix `ShallowCopy` is set for all `ShallowAuxContainer` type conta
 
 ## EDM manipulation
 
-The following configuration flag setting will override the "output" type for a job:
+The following configuration flag setting will override the "output level" for any AOD output file:
 ```python
 flags.Trigger.AODEDMSet='ESD'
 ```
-This will record all `ESD` labelled containers, even if the output is an AOD.
+i.e. this will record all `ESD` labelled containers to the output AOD file.
 
 
 The following configuration flag settings will override or add a new EDM entry if not already found:
@@ -149,23 +149,6 @@ Your container should appear in `AOD.pool.root.checkFile`. Each file provides de
 
 When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly or check the log file for the executed command. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
 
-For example, using an existing trigger test command file/log output as a reference, commands may look something like listed below.
-
-- To create an AOD on MC ttbar, `"Dev_pp_run3"` menu and `AODSLIM` target:
-```sh
-Reco_tf.py --outputAODFile=AOD.pool.root --steering "doRDO_TRIG" --CA "all:True"  --preExec "RDOtoRDOTrigger:flags.Exec.FPE=1" "all:flags.Trigger.triggerMenuSetup='Dev_pp_run3_v1';flags.Trigger.AODEDMSet='AODSLIM';" --preInclude "all:Campaigns.MC23e" --conditionsTag "default:OFLCOND-MC23-SDR-RUN3-08" --athenaopts="--imf --threads=1" --maxEvents=20 --inputRDOFile=root://eosatlas.cern.ch//eos/atlas/atlascerngroupdisk/data-art/large-input/trig-val/TriggerTest/valid1.601229.PhPy8EG_A14_ttbar_hdamp258p75_SingleLep.recon.RDO.e8514_e8528_s4369_s4370_r16083_tid42189392_00/RDO.42189392._000001.pool.root.1,root://eosatlas.cern.ch//eos/atlas/atlascerngroupdisk/data-art/large-input/trig-val/TriggerTest/valid1.601229.PhPy8EG_A14_ttbar_hdamp258p75_SingleLep.recon.RDO.e8514_e8528_s4369_s4370_r16083_tid42189392_00/RDO.42189392._000002.pool.root.1 >Reco_tf.log 2>&1
-```
-
-- To create an AOD on data, `"Dev_pp_run3"` menu and `AODFULL` target:
-```sh
-Trig_reco_tf.py --CA  --preExec "BSRDOtoRAW:Exec.FPE=1" "Trigger.triggerMenuSetup='Dev_pp_run3_v1_HLTReprocessing_prescale' Trigger.doLVL1=True" --prodSysBSRDO True --outputBSFile=RAW.pool.root --outputHIST_HLTMONFile=hltmon.root --outputDRAW_TRIGCOSTFile=TRIGCOST.pool.root --outputNTUP_TRIGCOSTFile=cost.ntup.root --runNumber 475321 --athenaopts="--imf --threads=4 --concurrent-events=4 --nprocs=1" --maxEvents=50 --inputBS_RDOFile=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-11._0001.1,/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-12._0001.1 >Trig_reco_tf.BSRDOtoRAW.log 2>&1
-
-Trig_reco_tf.py --inputBSFile=RAW.pool.root --outputAODFile=AOD.pool.root --outputNTUP_TRIGRATEFile=rate.ntup.root --outputHISTFile=ExampleMonitorOutput.root --conditionsTag="<conditions-tag>" --geometryVersion='<geometry-version>' --preExec="flags.Trigger.triggerMenuSetup='Dev_pp_run3_v1_HLTReprocessing_prescale'; flags.Trigger.AODEDMSet='AODFULL'; flags.DQ.Steering.doDataFlowMon=True; flags.DQ.Steering.doHLTMon=True; flags.DQ.Steering.doLVL1CaloMon=True; flags.DQ.Steering.doGlobalMon=True; flags.DQ.Steering.doLVL1InterfacesMon=True; flags.DQ.Steering.doCTPMon=True; flags.DQ.Steering.HLT.doBjet=True; flags.DQ.Steering.HLT.doInDet=True; flags.DQ.Steering.HLT.doBphys=True; flags.DQ.Steering.HLT.doCalo=True; flags.DQ.Steering.HLT.doEgamma=True; flags.DQ.Steering.HLT.doJet=True; flags.DQ.Steering.HLT.doMET=True; flags.DQ.Steering.HLT.doMinBias=True; flags.DQ.Steering.HLT.doMuon=True; flags.DQ.Steering.HLT.doTau=True;" --CA --athenaopts="--imf --threads=4 --concurrent-events=4" --maxEvents=50 >Trig_reco_tf.Tier0Reco.log 2>&1
-
-checkFile.py AOD.pool.root >AOD.pool.root.checkFile 2>&1;
-checkFileTrigSize.py AOD.pool.root >AOD.pool.root.checkFileTrigSize 2>&1
-```
-
 ### Cross-check: Container content
 
 First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag, e.g.:
-- 
GitLab


From a19eb1872844c204d1e49b5db7474dcecebf3718 Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Mon, 2 Jun 2025 13:56:08 +0200
Subject: [PATCH 6/9] Adding varToRemoveFromAODSLIM details and other edits

---
 docs/athena/trigger/developers/edm.md | 41 ++++++++++++---------------
 1 file changed, 18 insertions(+), 23 deletions(-)

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index 05b978a..dddb005 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -23,30 +23,33 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
 | Label | Output | Purpose |
 |-------------|-------------|-------------|
 |`BS`| ByteStream - Any content that is to make it out of Point 1 and be recorded to RAW file. | Required for recording to data AOD after or kept in BS for debugging in case of data taking issues.|
-|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies, also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
+|`ESD`| ESD files, RDO_TRIG files, SampleA productions | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies. Also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
 |`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning.|
 |`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors.|
 |`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information.|
 
-**Note**: SampleA trigger validation productions record all `ESD` labelled content.
+**Important**: Think hard about what targets you need, as the HLT content in total has been taking up a signficant fraction of disk space.
 
-**Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
+Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
 
-- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
-- Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+
+### Note on AODSLIM: Further reductions
+
+One can specify Run 3 container dynamic variables that are not needed in `AODSLIM`, i.e. variables that are not needed by physics analyses. This is set via the `varToRemoveFromAODSLIM` list in [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py).
 
 
 ## Optional EDM details
 
 There are several additional EDM details that can be specified as a list constituting the last entry in the tuple.
 
-### Views
+### InViews
 
 In the case of reconstruction in a limited region of the detector, this names the event view in which this collection is reconstructed, so that additional information allowing the items to be accessed in event views is recorded. For more details, see [the section on Event Views](eventviews.md).
 
 ### allowTruncation
 
-This is only valid for data taking. It is a flag to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
+This flag is only valid for data taking and is used to indicate that this container can be dropped from an event if the event data is larger than some threshold that would have resulted in the event data going to the debug stream. A developer need not worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content _before_ the last "allowTruncation" entries and, more specifically, within the relevant signature section.
 
 ### Aliases
 
@@ -93,8 +96,8 @@ The following steps should be followed if you would like to **add** new trigger
 1. Do the necessary software modications to record the new content as described in [section Reconstruction Algorithms: Saving EDM objects](algos.md#saving-edm-objects).
 1. Test and validate: Check content is recorded in the expected output (data AOD, MC AOD etc) and the content is correct. See the subsection below!
 1. Estimate the size increase. See the subsection below!
-1. Contact the trigger EDM coordinator [found on trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
-    (a) a JIRA ticket motivating the new implementation, new EDM content and EDM target, as well as a post of the results of the 2 previous steps.
+1. Contact the trigger EDM coordinator [found on the trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
+    (a) (likely already existing) JIRA ticket motivating the new implementation, in which is posted the motivation for the new EDM content and EDM target and the results of the 2 previous steps.
     (b) (if ready) a link to the draft Merge Request.
 1. The trigger EDM coordinator will confirm whether the EDM targets are appropriate and sufficiently motivated.
 1. If the change is implemented during a data taking period (e.g. Run 3) to the current Tier0 branch ({{data.branchTier0Short}}), the change requires "PROC" sign-off (Prompt Reconstruction Operation Coordinator) due to the _Frozen Tier0 Policy_ which tries to prevent EDM changes during data taking periods or major MC campaigns -- see details on the [FrozenTier0 twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy).
@@ -106,11 +109,10 @@ ERROR    Your change breaks the frozen tier0 policy in test q449.
 ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
 ```
 along with details of the diff. The failure is due to outdated AOD/ESD references, which PROC manages. Double-check that the differences are expected and tag the current PROC in your MR.
-The MR will only be merged once the ref updates have been done by PROC (once all tests are green). (TODO: Check - twiki says developers can update references themselves.)
+The MR will only be merged once the ref updates have been done by PROC (once all tests are green).
+1. It is likely that you will be requested to give a presentation of your new implementation at the Trigger General Meeting, in which case, provide 1-2 slides motivating the trigger EDM changes and size estimates.
 
-**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator also needs to, post-MR, update the bytestream schema, so that new container types can be serialised as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
-
-**Advice** Usually a new EDM implementation comes along with new trigger chains which need to be motivated in a presentation at the Trigger General Meeting if they are ready to go into the Physics menu. This presentation should also include material motivating the EDM content and the expected EDM size increase.
+**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised  as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
 
 ### Removing content
 
@@ -151,16 +153,9 @@ When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1Ph
 
 ### Cross-check: Container content
 
-First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag, e.g.:
-
-```sh
-athenaHLT.py -o output --imf --perfmon --threads=4 --concurrent-events=4 --nprocs=1 --number-of-events=50 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-11._0001.1 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-12._0001.1 TriggerJobOpts.runHLT Trigger.triggerMenuSetup="PhysicsP1_pp_run3_v1_HLTReprocessing_prescale" Trigger.selectChains=[\"HLT_j250_a10sd_cssk_ditauOmni0Trk99_pf_jes_ftf_preselj200_L1jJ160\",\"HLT_tau200_mediumGNTau_L1eTAU140\"] Trigger.doLVL1=True Exec.FPE=1 >athenaHLT.log 2>&1
-```
-TODO: This is for data. Easiest way to get skimmed MC on limited menu?
-
-This ensures that your output contains only events that passed your chain, meaning containers should be populated
-and ensures that all reconstruction is performed by _your_ chains
-(relevant if you are adding new chains that use already existing reconstructed objects).
+First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag.
+This ensures that all reconstruction and hence container content is the result of _your_ chains (relevant if you are adding new chains that write to already existing containers).
+Ideally, you run the trigger task on an input MC RDO file containing a good fraction of events that pass your triggers.
 
 Use an analysis script to retrieve the containers and plot variables to histogram or print to terminal to check that variables look correct.
 
-- 
GitLab


From 44d650160e15830dbbba0abdb39858b012e7cb99 Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Fri, 6 Jun 2025 18:12:00 +0200
Subject: [PATCH 7/9] minor update to trigger dev algos.md

---
 docs/athena/trigger/developers/algos.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/athena/trigger/developers/algos.md b/docs/athena/trigger/developers/algos.md
index c9a7df4..abfb3b4 100644
--- a/docs/athena/trigger/developers/algos.md
+++ b/docs/athena/trigger/developers/algos.md
@@ -49,7 +49,7 @@ The following software modifications are needed to record reconstructed objects
 1. in the python configuration of the algorithm an instance of the
   `recordable` function from [TrigEDMConfig.TriggerEDM]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDM.py#L34) needs
   to be used when setting the name of the collection. It assures that
-  the naming convention of the HLT EDM is followed and no trivial typos sneak in between name of the container .
+  the naming convention of the HLT EDM is followed and no trivial typos sneak in.
 1. The collection (type and name)  has to be listed in
   [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py) (or `TriggerEDMRun4` for upgrade studies)
   together with the required output file types in which it should be recorded. Typically the object should be stored
-- 
GitLab


From 44d147c1d0e914be6c9f250549797f1f4c37869a Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Fri, 6 Jun 2025 18:13:01 +0200
Subject: [PATCH 8/9] update trigger dev edm: edits + more detailed size
 estimate section

---
 docs/athena/trigger/developers/edm.md | 80 +++++++++++++++++++--------
 1 file changed, 57 insertions(+), 23 deletions(-)

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index dddb005..45a5385 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -55,6 +55,10 @@ This flag is only valid for data taking and is used to indicate that this contai
 
 A name with suffix `ShallowCopy` is set for all `ShallowAuxContainer` type containers to indicate that they need special treatment by the [HLTEDMCreator]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TrigSteer/TrigOutputHandling/src/HLTEDMCreator.h) given that the Aux store name differs from the interface name.
 
+## unit tests
+
+Unit tests exist that check the validity, ordering and manipulation of EDM entries.
+
 ## EDM manipulation
 
 The following configuration flag setting will override the "output level" for any AOD output file:
@@ -76,17 +80,6 @@ flags.Trigger.ExtraEDMList = [
 
 This command will update the targets for the already existing `HLT_TopoCaloClustersFS` and `HLT_TopoCaloClustersFSAux` containers, as well as add new `HLT_NewParticles` and `HLT_NewParticlesAux` containers.
 
-
-## unit tests
-
-TODO!
-
-Two unit tests exist that check for:
-
-- Validity of EDM entries.
-- Ordering of EDM entries.
-
-
 ## Procedure for modifying trigger EDM
 
 ### Adding content
@@ -112,7 +105,7 @@ along with details of the diff. The failure is due to outdated AOD/ESD reference
 The MR will only be merged once the ref updates have been done by PROC (once all tests are green).
 1. It is likely that you will be requested to give a presentation of your new implementation at the Trigger General Meeting, in which case, provide 1-2 slides motivating the trigger EDM changes and size estimates.
 
-**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised  as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
+**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised, as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
 
 ### Removing content
 
@@ -149,24 +142,65 @@ Your container should appear in `AOD.pool.root.checkFile`. Each file provides de
 
 #### Tips on running own trigger tests
 
-When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly or check the log file for the executed command. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
+When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
 
 ### Cross-check: Container content
 
-First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag.
+First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting `flags.Trigger.selectedChains=["my_new_chain"]`.
 This ensures that all reconstruction and hence container content is the result of _your_ chains (relevant if you are adding new chains that write to already existing containers).
-Ideally, you run the trigger task on an input MC RDO file containing a good fraction of events that pass your triggers.
+Ideally, you run the trigger task on an input MC RDO file containing a good fraction of events that pass your triggers. The ttbar MC RDO used in `test_trigAna_RDOtoAOD_v1MC_build.py` may be sufficient.
 
-Use an analysis script to retrieve the containers and plot variables to histogram or print to terminal to check that variables look correct.
+Use an analysis script to retrieve and plot EDM variables to histogram or open the AOD or RDO_TRIG file in interactive ROOT or ROOT browser to check that variable distributons look correct.
 
 ### Estimate size increase
 
-This can be tricky, especially if your chains have a low selection rate. A size estimate can be given by comparing between the latest Athena branch and your implementation:
-
-- Size of produced RAW (data) files between the current release and your implementation.
-- AOD file size difference quoted in AOD.pool.root.checkFile or summed size of new containers.
-- Trig EDM size difference quoted in AOD.pool.root.checkFileTrigSize.
+The size increase that primarily matters to us are increases in the the AOD ("AODSLIM" content) output from `test_trigAna_RDOtoAOD_v1MC_build` run on the MC menu, and the RAW and AOD ("AODFULL" content) otuput from `test_trigP1_v1PhysP1_T0Mon_build` run on the Physics menu.
+One can measure the size increase by comparing output from release {{data.branchTier0Short}} and a test physics menu with your trigger chains included/ by comparing file sizes or the numbers printed in `checkFile` and `checkFileTrigSize` output.
+A size increase can be quoted from comparing the sizes of the AOD/RAW output files: `AOD.pool.root` and `*.physics_Main*._athenaHLT*.data` and, for AOD, the sizes quoted in `AOD.pool.root.checkFile` (top of file) and `AOD.pool.root.checkFileTrigSize` (total trigger EDM size and for individual signature).
+
+```console
+du 24.0/AOD.pool.root
+4554	24.0/AOD.pool.root
+du my_dev_branch/AOD.pool.root
+4573	my_dev_branch/AOD.pool.root
+...
+grep triggerTau 24.0/AOD.pool.root.checkFileTrigSize
+triggerTau                       3.184
+grep triggerTau my_dev_branch/AOD.pool.root.checkFileTrigSize
+triggerTau                       4.067
+...
+grep "Total Trigger Size" 24.0/AOD.pool.root.checkFileTrigSize
+Total Trigger Size      51.414
+grep "Total Trigger Size" my_dev_branch/AOD.pool.root.checkFileTrigSize
+Total Trigger Size      52.311
+```
 
-TODO: Ran out of time. Needs more detail.
+One can also look at the absolute size of the new EDM additions in `AOD.pool.root.checkFile` and compare the "size/evt" to other similar containers or the total signature size. The "size/evt" for other containers can be retrieved from the "checkFile" output or from the [trigger EDM size monitoring webpage](https://test-atrvshft.web.cern.ch/test-atrvshft/TriggerEDMSizeMonitoring/webpage/) for the MC and PhysP1 tests mentioned above.
+
+```console
+================================================================================
+     Mem Size       Disk Size        Size/Evt      MissZip/Mem  items  (X) Container Name (X=Tree|Branch)
+================================================================================
+grep HLT_TrigTauRecMerged_MVA my_dev_branch/AOD.pool.root.checkFile
+       1.049 kb        0.256 kb        0.013 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVA
+       1.121 kb        0.316 kb        0.016 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_Loose
+       1.121 kb        0.316 kb        0.016 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_Tight
+       1.126 kb        0.317 kb        0.016 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_Medium
+       1.141 kb        0.324 kb        0.016 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_VeryLoose
+       1.939 kb        0.338 kb        0.017 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.viewIndex
+       1.426 kb        0.480 kb        0.024 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_Score
+       1.465 kb        0.484 kb        0.024 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAuxDyn.GNTau_ScoreSigTrans
+     144.458 kb       20.145 kb        1.007 kb        0.000       20  (B) HLT_TrigTauRecMerged_MVAAux.
+grep HLT_TrigTauRecMerged_LLP my_dev_branch/AOD.pool.root.checkFile
+       1.049 kb        0.256 kb        0.013 kb        0.000       20  (B) HLT_TrigTauRecMerged_LLP
+       1.541 kb        0.350 kb        0.017 kb        0.000       20  (B) HLT_TrigTauRecMerged_LLPAuxDyn.viewIndex
+      88.093 kb       12.688 kb        0.634 kb        0.000       20  (B) HLT_TrigTauRecMerged_LLPAux.
+grep HLT_TrigTauRecMerged_MVA my_dev_branch/AOD.pool.root.checkFile | awk '{SIZE=$5;SUM+=SIZE}END{print SUM}'
+1.149
+grep HLT_TrigTauRecMerged_LLP my_dev_branch/AOD.pool.root.checkFile | awk '{SIZE=$5;SUM+=SIZE}END{print SUM}'
+0.664
+grep triggerTau my_dev_branch/AOD.pool.root.checkFileTrigSize
+triggerTau                       4.067
+```
 
-Once the implementation is merged to Athena, ART (nightly) tests monitor the trigger EDM size on larger amount (~1000) of MC & data events. These numbers can be referenced in the JIRA. See [the section on trigger EDM size monitoring](validation.md#trigger-edm-size-monitoring) on obtaining the size estimates.
+It may be tricky to obtain a good size estimate if your triggers have a low selection rate. However, once your EDM changes are merged to the official Athena branch, ART (nightly) tests monitor the trigger EDM size on a larger amount (~1000) of MC & data events, which will give us a more accurate idea of the size increase. It is good to reference these numbers in your JIRA ticket once available (absolute "size/evt" values as well as relative size increase). See [the section on trigger EDM size monitoring](validation.md#trigger-edm-size-monitoring) on obtaining the size estimates.
-- 
GitLab


From 9795e3eef0b9ef5a4eb43d998650e0212bba3acd Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Tue, 10 Jun 2025 11:59:55 +0200
Subject: [PATCH 9/9] trigger dev edm: clarify, shorten

---
 docs/athena/trigger/developers/edm.md | 87 +++++++++++++++++++--------
 1 file changed, 61 insertions(+), 26 deletions(-)

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index 45a5385..94f5b76 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -90,22 +90,22 @@ The following steps should be followed if you would like to **add** new trigger
 1. Test and validate: Check content is recorded in the expected output (data AOD, MC AOD etc) and the content is correct. See the subsection below!
 1. Estimate the size increase. See the subsection below!
 1. Contact the trigger EDM coordinator [found on the trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
-    (a) (likely already existing) JIRA ticket motivating the new implementation, in which is posted the motivation for the new EDM content and EDM target and the results of the 2 previous steps.
-    (b) (if ready) a link to the draft Merge Request.
+    1. (likely already existing) JIRA ticket motivating the new implementation, in which is posted the motivation for the new EDM content and EDM target and the results of the 2 previous steps.
+    1. (if ready) a link to the draft Merge Request.
 1. The trigger EDM coordinator will confirm whether the EDM targets are appropriate and sufficiently motivated.
-1. If the change is implemented during a data taking period (e.g. Run 3) to the current Tier0 branch ({{data.branchTier0Short}}), the change requires "PROC" sign-off (Prompt Reconstruction Operation Coordinator) due to the _Frozen Tier0 Policy_ which tries to prevent EDM changes during data taking periods or major MC campaigns -- see details on the [FrozenTier0 twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy).
-If relevant, you should tag or email PROC after sign-off from the trigger group (FIXME: Check what actual procedure should be). You can find the current coordinators under the [ATLAS Data Preparation organisation page](https://atlaspo.cern.ch/public/ATLASOrganisation/index.html?value=DATAPREP) or on the twiki.
-1. Once the MR is undrafted, there are a couple of CI tests that may fail due to breaking the Frozen Tier0 policy during data taking, namely `CITest_RecoRun3Data_Checks-test` and `CITest_RecoRun3MC-test`. The ERROR will include
+1. Submit/undraft the MR
+    1. If an MR changes the EDM during a data taking period (e.g. Run 3) to the current Tier0 branch ({{data.branchTier0Short}}), the MR will require "PROC" sign-off (Prompt Reconstruction Operation Coordinator) due to the _Frozen Tier0 Policy_ which tries to prevent EDM changes during data taking periods or major MC campaigns -- see details on the [FrozenTier0 twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy). There will be failing CI tests due to this policy, namely `CITest_RecoRun3Data_Checks-test` and `CITest_RecoRun3MC-test`:
 ```sh
 ERROR    Py:diff-root        ERROR files differ!
 ERROR    Your change breaks the frozen tier0 policy in test q449.
 ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
 ```
-along with details of the diff. The failure is due to outdated AOD/ESD references, which PROC manages. Double-check that the differences are expected and tag the current PROC in your MR.
-The MR will only be merged once the ref updates have been done by PROC (once all tests are green).
-1. It is likely that you will be requested to give a presentation of your new implementation at the Trigger General Meeting, in which case, provide 1-2 slides motivating the trigger EDM changes and size estimates.
+along with details of the diff. **Double-check that the differences are expected** and tag the current PROC in your MR, providing the JIRA link with motivation (see [ATLAS Data Preparation organisation page](https://atlaspo.cern.ch/public/ATLASOrganisation/index.html?value=DATAPREP) or [twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy) for current coordinators).
+The MR will only be merged once the AOD/ESD references have been updated with your changes by PROC (once all tests are green).
+1. When requested to give a presentation of your new implementation at the Trigger General Meeting, provide 1-2 slides motivating the trigger EDM changes and size estimates.
 
-**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised, as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
+!!! note
+    Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised, as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
 
 ### Removing content
 
@@ -116,16 +116,13 @@ If you would like to **remove** trigger EDM content, then get in touch with the
 The following should be cross-checked for new EDM content:
 
 1. New EDM containers appear in RAW or relevant AOD output.
-1. New EDM containers are populated for events that seed the relevant reconstruction (this is not only events where the relevant chain has passed).
-1. Dynamic values can be accessed and make sense: Are not default values, zeroed, etc.
-1. The size increase is as expected and not exhorbitant.
+1. New EDM containers are populated by your trigger reconstruction (Note: Containers will be filled with the online reconstruction for testing your trigger hypothesis, even if your trigger didn't pass).
+1. Dynamic values can be accessed and make sense: Are not default values, etc.
+1. The size increase is as expected and not exorbitant.
 
 The results should be documented in JIRA.
 
-!!! warning
-    All EDM containers listed in `TriggerEDMRunX` _will_ appear in the relevant output specified by the EDM target, _regardless_ if they were reconstructed at the HLT: We run a "GapFiller" each time we process produce AODs, which creates empty HLT containers for an event if they don't already exist to ensure a consistent EDM throughout all events. Thus, seeing a container listed doesn't mean it's populated!
-
-### Cross-check: Container recording
+#### Useful trigger tests
 
 The following tests will output AODs after rerunning the HLT on a given menu:
 
@@ -136,33 +133,70 @@ The following tests will output AODs after rerunning the HLT on a given menu:
 | test_trigP1_v1PhysP1_T0Mon_build | data | AODFULL | `"Physics_pp_run3"`|
 
 Each test will output `AOD.pool.root.checkFile` and `AOD.pool.root.checkFileTrigSize` files, produced from
-`checkFile.py AOD.pool.root` and `checkFileTrigSize.py AOD.pool.root`, respectively.
+`checkFile.py AOD.pool.root` and `checkFileTrigSize.py AOD.pool.root`, respectively: `checkFile` output provides sizes per container element, `checkFileTrigSize` provides EDM sizes for the total _trigger_ EDM as well as individual trigger signatures.
 
-Your container should appear in `AOD.pool.root.checkFile`. Each file provides details on container or total component sizes.
 
 #### Tips on running own trigger tests
 
-When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
+When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py` or `test_trigP1_v1PhysP1_T0Mon_build.py`, files are automatically created containing the executables: `commands.json`,  as well as `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
+
+### Cross-check: Container recording
+
+Your container should appear in `AOD.pool.root.checkFile` for a relevant ("AODFULL", "AODSLIM" ...) AOD file.
+
+!!! warning
+    All EDM containers listed in `TriggerEDMRunX` _will_ appear in the relevant output specified by the EDM target, _regardless_ if they were reconstructed at the HLT: We run a "GapFiller" each time we produce AODs, which creates empty HLT containers for an event if they don't already exist to ensure a consistent EDM throughout all events. Thus, seeing a container listed doesn't mean it's populated!
 
 ### Cross-check: Container content
 
-First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting `flags.Trigger.selectedChains=["my_new_chain"]`.
-This ensures that all reconstruction and hence container content is the result of _your_ chains (relevant if you are adding new chains that write to already existing containers).
+You should verify that new containers _are_ being filled and that they are populated by _your_ chains. A very useful cross-check is to run a trigger test on a limited menu containing your chains only by setting `flags.Trigger.selectedChains=["my_new_chain"]`.
+This ensures that all reconstruction and hence container content is the result of _your_ chains (especially relevant if you are adding new chains that write to already existing containers).
 Ideally, you run the trigger task on an input MC RDO file containing a good fraction of events that pass your triggers. The ttbar MC RDO used in `test_trigAna_RDOtoAOD_v1MC_build.py` may be sufficient.
 
 Use an analysis script to retrieve and plot EDM variables to histogram or open the AOD or RDO_TRIG file in interactive ROOT or ROOT browser to check that variable distributons look correct.
 
 ### Estimate size increase
 
-The size increase that primarily matters to us are increases in the the AOD ("AODSLIM" content) output from `test_trigAna_RDOtoAOD_v1MC_build` run on the MC menu, and the RAW and AOD ("AODFULL" content) otuput from `test_trigP1_v1PhysP1_T0Mon_build` run on the Physics menu.
-One can measure the size increase by comparing output from release {{data.branchTier0Short}} and a test physics menu with your trigger chains included/ by comparing file sizes or the numbers printed in `checkFile` and `checkFileTrigSize` output.
-A size increase can be quoted from comparing the sizes of the AOD/RAW output files: `AOD.pool.root` and `*.physics_Main*._athenaHLT*.data` and, for AOD, the sizes quoted in `AOD.pool.root.checkFile` (top of file) and `AOD.pool.root.checkFileTrigSize` (total trigger EDM size and for individual signature).
+The size increase that primarily matters to us are for:
+
+-  `test_trigAna_RDOtoAOD_v1MC_build` AOD output ("AODSLIM" content), run on the MC menu
+-  `test_trigP1_v1PhysP1_T0Mon_build` RAW and AOD output ("AODFULL" content), run on the Physics menu.
+
+It is useful to have the size increase quoted as percentage relative to: total file size, total trigger EDM size (from `checkFileTrigSize`) and relevant trigger signature size (from `checkFileTrigSize`).
+
+One can measure the size increase by:
+
+- Comparing output file sizes (e.g. `AOD.pool.root`, `*.physics_Main*._athenaHLT*.data`) between release {{data.branchTier0Short}} and development branch with your trigger chains included in the physics menu.
+- Comparing numbers printed in `checkFile` and `checkFileTrigSize` output: e.g. `AOD.pool.root.checkFile` provides total file size + size for individual EDM elements, `AOD.pool.root.checkFileTrigSize` provides sizes for total trigger EDM and for individual signatures.
+
+Example terminal output comparing total file, trigger EDM and individual trigger signature sizes:
 
 ```console
 du 24.0/AOD.pool.root
 4554	24.0/AOD.pool.root
 du my_dev_branch/AOD.pool.root
 4573	my_dev_branch/AOD.pool.root
+...
+head 24.0/AOD.pool.root.checkFile
+PyRoot: Styles set by rootlogin.C
+## opening file [AOD.pool.root]...
+## importing ROOT...
+## importing ROOT... [DONE]
+## opening file [OK]
+File:AOD.pool.root
+Size:     4553.341 kb
+Nbr Events: 20
+
+head my_dev_branch/AOD.pool.root.checkFile
+PyRoot: Styles set by rootlogin.C
+## opening file [AOD.pool.root]...
+## importing ROOT...
+## importing ROOT... [DONE]
+## opening file [OK]
+File:AOD.pool.root
+Size:     4572.145 kb
+Nbr Events: 20
+
 ...
 grep triggerTau 24.0/AOD.pool.root.checkFileTrigSize
 triggerTau                       3.184
@@ -175,7 +209,7 @@ grep "Total Trigger Size" my_dev_branch/AOD.pool.root.checkFileTrigSize
 Total Trigger Size      52.311
 ```
 
-One can also look at the absolute size of the new EDM additions in `AOD.pool.root.checkFile` and compare the "size/evt" to other similar containers or the total signature size. The "size/evt" for other containers can be retrieved from the "checkFile" output or from the [trigger EDM size monitoring webpage](https://test-atrvshft.web.cern.ch/test-atrvshft/TriggerEDMSizeMonitoring/webpage/) for the MC and PhysP1 tests mentioned above.
+Example terminal output looking at the "Size/Evt" of individual EDM additions in `AOD.pool.root.checkFile`and comparing to other similar containers - the "Size/Evt" for other containers can be retrieved from the "checkFile" output if run on the full MC/Physics menu or from the [trigger EDM size monitoring webpage](https://test-atrvshft.web.cern.ch/test-atrvshft/TriggerEDMSizeMonitoring/webpage/) for the MC and PhysP1 tests mentioned above:
 
 ```console
 ================================================================================
@@ -203,4 +237,5 @@ grep triggerTau my_dev_branch/AOD.pool.root.checkFileTrigSize
 triggerTau                       4.067
 ```
 
-It may be tricky to obtain a good size estimate if your triggers have a low selection rate. However, once your EDM changes are merged to the official Athena branch, ART (nightly) tests monitor the trigger EDM size on a larger amount (~1000) of MC & data events, which will give us a more accurate idea of the size increase. It is good to reference these numbers in your JIRA ticket once available (absolute "size/evt" values as well as relative size increase). See [the section on trigger EDM size monitoring](validation.md#trigger-edm-size-monitoring) on obtaining the size estimates.
+!!! note
+    It can be difficult obtaining a good size estimate if your triggers have a low selection rate. However, once your EDM changes are merged to the official Athena branch, ART (nightly) tests monitor the trigger EDM size on a larger amount (~1000) of MC & data events, which will give us a more accurate idea of the size increase. It is good to reference these numbers in your JIRA ticket once available (absolute "size/evt" values as well as relative size increase). See [the section on trigger EDM size monitoring](validation.md#trigger-edm-size-monitoring) on obtaining these size estimates.
-- 
GitLab