From 61bedd0ae4f304818cbb4f26b17b6d0caf65771b Mon Sep 17 00:00:00 2001
From: cantel <claire.antel@gmail.com>
Date: Mon, 2 Jun 2025 13:56:08 +0200
Subject: [PATCH] Adding varToRemoveFromAODSLIM details and other edits

---
 docs/athena/trigger/developers/edm.md | 41 ++++++++++++---------------
 1 file changed, 18 insertions(+), 23 deletions(-)

diff --git a/docs/athena/trigger/developers/edm.md b/docs/athena/trigger/developers/edm.md
index 05b978a..dddb005 100644
--- a/docs/athena/trigger/developers/edm.md
+++ b/docs/athena/trigger/developers/edm.md
@@ -23,30 +23,33 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
 | Label | Output | Purpose |
 |-------------|-------------|-------------|
 |`BS`| ByteStream - Any content that is to make it out of Point 1 and be recorded to RAW file. | Required for recording to data AOD after or kept in BS for debugging in case of data taking issues.|
-|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies, also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
+|`ESD`| ESD files, RDO_TRIG files, SampleA productions | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies. Also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
 |`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning.|
 |`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors.|
 |`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information.|
 
-**Note**: SampleA trigger validation productions record all `ESD` labelled content.
+**Important**: Think hard about what targets you need, as the HLT content in total has been taking up a signficant fraction of disk space.
 
-**Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
+Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
 
-- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
-- Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
+
+### Note on AODSLIM: Further reductions
+
+One can specify Run 3 container dynamic variables that are not needed in `AODSLIM`, i.e. variables that are not needed by physics analyses. This is set via the `varToRemoveFromAODSLIM` list in [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py).
 
 
 ## Optional EDM details
 
 There are several additional EDM details that can be specified as a list constituting the last entry in the tuple.
 
-### Views
+### InViews
 
 In the case of reconstruction in a limited region of the detector, this names the event view in which this collection is reconstructed, so that additional information allowing the items to be accessed in event views is recorded. For more details, see [the section on Event Views](eventviews.md).
 
 ### allowTruncation
 
-This is only valid for data taking. It is a flag to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
+This flag is only valid for data taking and is used to indicate that this container can be dropped from an event if the event data is larger than some threshold that would have resulted in the event data going to the debug stream. A developer need not worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content _before_ the last "allowTruncation" entries and, more specifically, within the relevant signature section.
 
 ### Aliases
 
@@ -93,8 +96,8 @@ The following steps should be followed if you would like to **add** new trigger
 1. Do the necessary software modications to record the new content as described in [section Reconstruction Algorithms: Saving EDM objects](algos.md#saving-edm-objects).
 1. Test and validate: Check content is recorded in the expected output (data AOD, MC AOD etc) and the content is correct. See the subsection below!
 1. Estimate the size increase. See the subsection below!
-1. Contact the trigger EDM coordinator [found on trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
-    (a) a JIRA ticket motivating the new implementation, new EDM content and EDM target, as well as a post of the results of the 2 previous steps.
+1. Contact the trigger EDM coordinator [found on the trigger Core s/w twiki](https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerCoreSoftware), providing:
+    (a) (likely already existing) JIRA ticket motivating the new implementation, in which is posted the motivation for the new EDM content and EDM target and the results of the 2 previous steps.
     (b) (if ready) a link to the draft Merge Request.
 1. The trigger EDM coordinator will confirm whether the EDM targets are appropriate and sufficiently motivated.
 1. If the change is implemented during a data taking period (e.g. Run 3) to the current Tier0 branch ({{data.branchTier0Short}}), the change requires "PROC" sign-off (Prompt Reconstruction Operation Coordinator) due to the _Frozen Tier0 Policy_ which tries to prevent EDM changes during data taking periods or major MC campaigns -- see details on the [FrozenTier0 twiki](https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/FrozenTier0Policy).
@@ -106,11 +109,10 @@ ERROR    Your change breaks the frozen tier0 policy in test q449.
 ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
 ```
 along with details of the diff. The failure is due to outdated AOD/ESD references, which PROC manages. Double-check that the differences are expected and tag the current PROC in your MR.
-The MR will only be merged once the ref updates have been done by PROC (once all tests are green). (TODO: Check - twiki says developers can update references themselves.)
+The MR will only be merged once the ref updates have been done by PROC (once all tests are green).
+1. It is likely that you will be requested to give a presentation of your new implementation at the Trigger General Meeting, in which case, provide 1-2 slides motivating the trigger EDM changes and size estimates.
 
-**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator also needs to, post-MR, update the bytestream schema, so that new container types can be serialised as well as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
-
-**Advice** Usually a new EDM implementation comes along with new trigger chains which need to be motivated in a presentation at the Trigger General Meeting if they are ready to go into the Physics menu. This presentation should also include material motivating the EDM content and the expected EDM size increase.
+**NOTE** Notifying the Trigger EDM coordinator is very important. Besides reviewing size increases, the trigger EDM coordinator may also need to, post-MR, update the bytestream schema, so that new container types can always be serialised  as ensure that Athena releases for P1/Tier0 and for GlobalMonitoring are synchronised!
 
 ### Removing content
 
@@ -151,16 +153,9 @@ When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1Ph
 
 ### Cross-check: Container content
 
-First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag, e.g.:
-
-```sh
-athenaHLT.py -o output --imf --perfmon --threads=4 --concurrent-events=4 --nprocs=1 --number-of-events=50 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-11._0001.1 --file=/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/TrigP1Test/data24_13p6TeV.00475321.physics_EnhancedBias.merge.RAW._lb0231._SFO-12._0001.1 TriggerJobOpts.runHLT Trigger.triggerMenuSetup="PhysicsP1_pp_run3_v1_HLTReprocessing_prescale" Trigger.selectChains=[\"HLT_j250_a10sd_cssk_ditauOmni0Trk99_pf_jes_ftf_preselj200_L1jJ160\",\"HLT_tau200_mediumGNTau_L1eTAU140\"] Trigger.doLVL1=True Exec.FPE=1 >athenaHLT.log 2>&1
-```
-TODO: This is for data. Easiest way to get skimmed MC on limited menu?
-
-This ensures that your output contains only events that passed your chain, meaning containers should be populated
-and ensures that all reconstruction is performed by _your_ chains
-(relevant if you are adding new chains that use already existing reconstructed objects).
+First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag.
+This ensures that all reconstruction and hence container content is the result of _your_ chains (relevant if you are adding new chains that write to already existing containers).
+Ideally, you run the trigger task on an input MC RDO file containing a good fraction of events that pass your triggers.
 
 Use an analysis script to retrieve the containers and plot variables to histogram or print to terminal to check that variables look correct.
 
-- 
GitLab