The following software modifications are needed to record reconstructed objects created by HLT algorithms to file:
1. in the python configuration of the algorithm an instance of the
`recordable` function from [TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py#L34) needs
`recordable` function from [TrigEDMConfig.TriggerEDM]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDM.py#L34) needs
to be used when setting the name of the collection. It assures that
the naming convention of the HLT EDM is followed and no trivial typos sneak in between name of the container .
1. The collection (type and name) has to be listed in
[TrigEDMConfig.TriggerEDMRun3]({{data.athena_git_url}}/blob/{{data.branchTier0Short}}/Trigger/TriggerCommon/TrigEDMConfig/python/TriggerEDMRun3.py)(or`TriggerEDMRun4` for upgrade studies)
together with the required output file types in which it should be recorded. Typically the object should be stored
at least in `BS` and `ESD`.
at least in `BS` and `ESD` (`RDO_TRIG` files - the equivalent of "raw" files for MC - contain all `ESD`-labelled content).
See the [trigger EDM guidelines](edm.md#procedure-for-modifying-trigger-edm) on the full procedure for adding new trigger EDM content (testing, verification and approval).
## Troubleshooting
Typically a reconstruction algorithm uses many input data
@@ -13,8 +13,8 @@ Each EDM entry is a tuple containing all EDM details for a specific interface or
```
Every EDM entry will at minimum define the container type and container name, the "EDM targets" and trigger signature. Any additional dynamic Aux variables
to be recorded are specified as part of the Aux container name string.
The EDM targets define the output type to which the containers will be recorded. We use these to restrict the HLT content when not all HLT content is needed,
e.g. in MC samples for analysis. The trigger signature is used to categorise total EDM size by signature in the Trigger EDM monitoring. (FIXME REF)
The EDM targets define the output type to which the containers will be recorded in the case of `RAW`, `RDO_TRIG`, `ESD` and `AOD` files, as well as PHYSVAL `DAOD` files. The `ESD` level is used for MC `RDO_TRIG` (equivalent to data `RAW` files for MC) production. We use these to restrict the HLT content when not all HLT content is needed,
e.g. in MC AODs for physics analysis derivations. The trigger signature is used to categorise total EDM size by signature in the [Trigger EDM monitoring](validation.md#trigger-edm-size-monitoring).
## EDM targets
...
...
@@ -23,16 +23,16 @@ The EDM target labels (2nd entry in an EDM entry) are mapped to the following ou
| Label | Output | Purpose |
|-------------|-------------|-------------|
|`BS`| ByteStream - Any content that is to make it out of Point 1 and be recorded to RAW file. | Required for recording to data AOD after or kept in BS for debugging in case of data taking issues.|
|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies.|
|`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning.|
|`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors.|
|`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information.|
|`ESD`| ESD files | Low-level reconstructed information such as HLT calorimeter clusters for specialised reconstruction studies, also used for `RDO_TRIG` (MC equivalent of a RAW file) output.|
|`AODFULL`| data AOD at Tier0, trigger developer AOD production requests, HLT Reprocessing AODs. | Required for regular trigger efficiency studies in data or validation during commissioning.|
|`AODSLIM`| bulk MC production AODs for physics analysis, SampleT productions. | Required for physics analysis. Examples: Final precision HLT leptons/b-jets needed for trigger matching and trigger scale factors.|
|`PhysicsTLA`/`EgammaPEBTLA`/`DarkJetPEBTLA`/`FTagPEBTLA`| respective RAW output of a Trigger-Level Analysis (TLA) or TLA Partial Event Building stream| Content required for a trigger-level physics analysis or TLA with partial event building (PEB) information.|
**Note**: SampleA trigger validation productions record all `ESD` labelled content.
**Important**: Think hard about what targets you need. The HLT content in total constitutes 25% of a physics_Main AOD and 20% of a RAW file, and we have far outblown our initial pledge for Run 3 disk space.
- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in data or if it is a new implementation that needs a few months of commissioning.
- Valid reasons to keep trigger content in `AODFULL`: If needed for regular trigger efficiency checks in recorded data AOD or if it is a new implementation that needs a few months of commissioning.
- Valid reasons to keep trigger content in `AODSLIM`: If needed for physics analysis, namely trigger matching and scale factor derivations.
...
...
@@ -42,11 +42,11 @@ There are several additional EDM details that can be specified as a list constit
### Views
This names the event view in which this collection is reconstructed in, in the case of reconstruction in a limited region of the detector, so that additionally information allowing the items to be accessed in an event view, is recorded. For more details, see [the section on EventViews](eventviews.md).
In the case of reconstruction in a limited region of the detector, this names the event view in which this collection is reconstructed, so that additional information allowing the items to be accessed in event views is recorded. For more details, see [the section on EventViews](eventviews.md).
### allowTruncation
This is only valid for data taking. It is an object to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, ""allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
This is only valid for data taking. It is a flag to indicate that this result can be dropped from an event, if the event data is larger than some threshold, which would have otherwise caused it to go into the debug stream. It is not necessary for a developer to worry about setting this EDM detail. However, for technical reasons, "allowTruncation" entries need to appear at the end of the EDM list. Therefore, a developer should add any new EDM content before the last "allowTruncation" entries - preferably within the relevant signature section.
### Aliases
...
...
@@ -54,11 +54,11 @@ A name with suffix `ShallowCopy` is set for all `ShallowAuxContainer` type conta
## EDM manipulation
The following configuration flag setting will override the "output" type for a job:
The following configuration flag setting will override the "output level" for any AOD output file:
```python
flags.Trigger.AODEDMSet='ESD'
```
This will record all `ESD` labelled containers, even if the output is an AOD.
i.e. this will record all `ESD` labelled containers to the output AOD file.
The following configuration flag settings will override or add a new EDM entry if not already found:
...
...
@@ -149,23 +149,6 @@ Your container should appear in `AOD.pool.root.checkFile`. Each file provides de
When you run tests like `test_trigAna_RDOtoAOD_v1MC_build.py`, `test_trigP1_v1PhysP1_T0Mon_build.py` or `test_trigP1_v1PhysP1_T0MonTrf_build.py`, files are automatically created containing the executables: `command.json` for the former (MC reco) and `runargs.RAWtoALL.py` for the latter. Alternatively, you copy and edit the test directly or check the log file for the executed command. You can use this output as a reference for running your own trigger test with modified settings (different menu, number of events, EDM target etc).
For example, using an existing trigger test command file/log output as a reference, commands may look something like listed below.
- To create an AOD on MC ttbar, `"Dev_pp_run3"` menu and `AODSLIM` target:
First, you want to check that your containers are populated by your chains. A very useful cross-check is to run a trigger test with only your chains present by setting the `Trigger.selectedChains=["chain1", "chain2"]` flag, e.g.:
