Hello,

This MR is deploying the final MVA Tau Energy Scale calibration for the R22 Run2 reprocessing.
It was derived on Round 3 MC preproduction (ATLTAU-1785).
In the previous iteration (Round 2.5), the tau track classifier and the tau energy scale were tuned separately, while the energy scale depends on track classification.
So we needed to update MVA TES using Round 3 MC that has the final track classifier tune.
The updated MVA TES is better behaved (very slightly better resolution, and the double-Gaussian structure that was seed in the energy resolution at very high pt is now gone).

One input variable of the 0-prong boosted regression tree was seen to be useless and was dropped (it is now only used for the >=1 prong MVA TES BRT).

The thinning cuts (pt cuts above which we write taus to AOD) were adjusted to get the same thinning efficiencies vs pt as with Round 2.5 MC, such that the AOD size is almost unchanged.
From SPOT tests, the total event size changes from 342.391 kb to 343.225 kb for ttbar MC, and from 399.221 kb to 398.751 kb for data17.

I also update the digest counts: as one could expect, we observe small changes in the number of ID tracks, due to TRT standalone tracks being thinned from AODs if they are not used by egamma nor tau reconstruction (and tau thinning is pt-dependent, thus depends on the MVA TES tune).

The MR was tested in tau standalone reco from ESD, q221, and xAOD digest.

I put the urgent flag because we would need this for the reprocessing.

Cheers,
Bertrand

Hello,

This MR is deploying the final Tau Energy Scale calibrations for combined TES (ATLTAU-1740) and MVA TES (ATLTAU-1739).
The tunes were derived on Round 2.5 gamma* MC preproduction.

For MVA TES, two small changes were made to the variable definitions.

* First, ptDetectorAxis was replaced with the proper tau-vertex-corrected LC-scale tau pt, namely ptIntermediateAxis.
These two pt calibrations are very close, but we should be using ptIntermediateAxis.
* Second, for BDT scores that come out of the tau substructure reconstruction (BDTs that separate 1p0n/1p1n/1pXn and 3p0n/3pXn modes), something had been overlooked.
The BDT scores are initialised to a default value of -1111 in the reconstruction, while the meaningful values set in the substructure reconstruction are within [-5,1].
So having a range that spans [-1111, 1] is quite bad for the MVA TES BRT optimisation, basically the cuts on these variable seemed completely random (TMVA probably scans for optimal cuts with a fixed step size).
Scanning values in the meaningful range [-5-epsilon, 1] makes much more sense, although the BRT performance does not improve much.

I'm also updating the digest counts, as tau energy calibration is known to impact the number of tracks in AODs, via tau track thinning.

Adding the urgent flag as the MR is needed for the R22 reprocessing.

Cheers,
Bertrand

Hello,

This MR adds a protection in the calculation of a weight that is meant to be within [0,1], but can be <0 or >1 in rare cases.
The weight is used in the calculation of the weighted average of calo-only tau pt and substructure tau pt.
This MR changes the tau reconstruction output in rare cases.

I also take this opportunity to deploy the final R22 calo-only calibration file tuned on Round 2.5 MC preproduction.

Cheers,
Bertrand

Hello,

This MR is preparing the ground for R22 RNN tau ID, which we expect to deploy in the next days (ATLTAU-1741).
For now, we use dummy calibration files for 2p (the same files as for 3p, such that the reconstruction output is unchanged).

We also introduce new variables that will be used in the R22 RNN for tracks and clusters.
Some are close to R21 equivalents (track impact parameters: log vs no log), but the trigger still relies on R21 variable definitions, so we cannot decommission the R21 variables yet.

The next MR will just push new calibration files (new json files for the RNN, new ROOT files for the working points).

Cheers,
Bertrand

Hello,

As discussed in ATLPHYSVAL-715, this MR is deploying a new default calibration for the RNN tau track classifier.
This is the latest available tuning that was deployed for Round2 preproduction already via a preExec.
The goal of this MR is just to avoid specifying this in future preExecs.
But it's a temporary tune, it will be superseded in a couple of weeks when we have the final tune for the R22 reprocessing.

The change was tested in tau standalone reco from ESD (and was anyway used for Round2 MC preprod).

Cheers,
Bertrand

Hello,

This MR is a follow-up of !44662 that introduced ghost matching in the tau-track association procedure (ATLTAU-1442).
Ghost matching is now activated by default in the offline tau reconstruction (won't be used in the trigger).

In addition, tracks beyond R=0.25 of the tau axis and which are not ghost matched are now kept if the jet closest to the track is the seed jet of the current tau.
This condition was introduced to recover tau-track association efficiency for low-pt 3p taus, where tracks from the tau decay are not always ghost-matched with the seed jet.
This scheme will be used for the R22 reprocessing after it is carefully validated.

Sorry for adding the urgent flag again, but it would be great to have this MR reviewed and merged quickly, as it will be needed for the final Round of algorithm tuning for the R22 reprocessing.

Cheers and thanks,
Bertrand

Hello,

This MR is deploying a preliminary version of the combined Tau Energy Scale calibration for R22 (ATLTAU-1740), tuned on Round1+Round2 MC preproduction samples.
It uses a compatibility condition between calo-only pt and substructure pt to reduce non-Gaussian tails from substructure pt.
So far, the compatibility condition was the same for all decay modes, but it now depends on the decay mode.
The calibration is preliminary (the ROOT file is on EOS, soon on cvmfs), and a final tune will be deployed for the R22 reprocessing.

Cheers,
Bertrand

Hello,

This MR is implementing a hybrid tau-track association that accepts all tracks within a cone of radius 0.25, and beyond 0.25, tracks are required to be ghost-matched with the seed jet.
The ghost matching requirement prevents from using tracks from another tau (a situation that leads to very problematic behaviour of tau substructure reconstruction, ATLTAU-1442).
The motivations are explained in [this talk](https://indico.cern.ch/event/1051117/contributions/4420815/attachments/2269038/3853037/TauCP_Martin_22Jun21.pdf)

The hybrid cone/ghost scheme is meant to replace the R=0.4 cone association, but it's currently disabled in tauRec.

I am adding the urgent flag because we need to validate the new scheme urgently and use it for upcoming retunings of tau algorithms for the R22 reprocessing.

Cheers,
Bertrand

Hello,

This MR is implementing a configuration for jet seeding + tau thinning that is closer to what we will use in the R22 reprocessing, namely:

* no JES-calibrated pt cut on the AntiKt4LCTopoJets used to seed the tau reconstruction
* a tau pt cut applied to the MVA tau pt at the end of the tau reconstruction, to thin the tau containers before writing them out to AOD

The tau thinning cut will be fine tuned, for now it's a simple 10 GeV pt cut.
This will lead to a significant increase in CPU as we will now run the tau reconstruction on all seed jets, not only on those with pt>15 GeV.
The impact on the AOD size depends on the MVA energy resolution and other factors that are still being tuned.
We expect to see changes in SPOT plots (ATLASRECTS-5684).

Cheers,
Bertrand

Hello,

This MR deploys a first retuning of MVA tau energy scale calibration derived on R22 Round2 MC preproduction (ATLTAU-1739).
This won't be the final tune, as we need to tune 2 intermediate TES calibration upstream before getting the final MVA TES tune.
But we need a close-to-final baseline for xAOD size studies, and it will also be helpful for other algorithm tunings that depend on MVA TES (e.g. RNN tau ID WPs).
The variable names were updated to better reflect their definition.

For completeness:

* training on 5M taus
* EventInfoAux.eventNumber%2 == 0 && TauJetsAuxDyn.nTracks<6
* NTrees=2500:BoostType=Grad:Shrinkage=0.1:UseBaggedBoost:BaggedSampleFraction=0.5:nCuts=250:MaxDepth=5

Cheers,
Bertrand

Bertrand Martin Dit Latour authored 3 years ago and Adam Edward Barton committed 3 years ago

Hello,

This MR is deploying the preliminary R22 tunes for calo TES (ATLTAU-1744) and MVA TES (ATLTAU-1739) derived on Round1 MC pre-prod samples.
For calo TES, it's only a change in calibration file.
For MVA TES, to support the new calibration file, we need to add the 2 new input variables for the BRT: the median event energy density rho, and the ratio between EM and LC scale pt.
That results in 2 new decorations added onto the tau, "rho" and "ptIntermediateAxisEM". The former is kept in the xAOD output for the time being, to ease the validation, but will be dropped for the reprocessing.

Cheers,
Bertrand

Bertrand Martin Dit Latour authored 3 years ago and Adam Edward Barton committed 3 years ago

Hello,

This MR deploys the R22 preliminary tune for RNN track classification derived on Round1 MC pre-prod samples (ATLTAU-1738).
The RNN uses additional variables, some related to the primary vertex, which necessitates to read in the PrimaryVertices container, and pass it on to the downstream tool that actually evaluates the RNN inputs.
Some minor code cleanup and reorganisation was also made.

Adding the urgent flag as we need this MR for Round2 MC pre-production to be launched very soon.
We might readjust the input variables for the Run2 reprocessing, based on final studies to be made on Round2 samples.

Cheers,
Bertrand

Hello,

This MR is defining a few more tauRec flags to avoid hardcoding calibration files in TauAlgorithmsHolder. It makes it easier to change the files via a preExec.
We also move the default calibration path to "R22_preprod", anticipating the upcoming MRs that will deploy the R22 preliminary tunes to be used for Round2 MC pre-production.

Cheers,
Bertrand

Hello,

This MR is instructing tauRec to set the calibration path for all tools (even for tools that don't use calibration files) according to the tauRecToolsCVMFSPath flag.
Currently, if not specified explicitly in TauAlgorithmsHolder, we are picking up some default path set in TauRecToolBase (the default has been changed to an empty string, to make sure we don't pick a wrong default value), and the path must be set for each tool individually, which is error prone.

The only exception that still remains is the RNN tau track classifier, where the path is set for the tool of a tool, rather than the tool directly. This is to be improved.

I also cleaned up deprecated tool configurations and flags that will no longer be used in R22.

This MR does not change the reconstruction output.

Cheers,
Bertrand

Hello,

This MR is changing the seed jet pt cut from 10 GeV to 0 GeV in the tau reconstruction.
It does not change the reconstruction output, because this cut is set upstream to 15 GeV in the jet reconstruction: https://gitlab.cern.ch/atlas/athena/-/blob/master/Reconstruction/Jet/JetRec/share/JetRec_jobOptions.py#L33.
However, for the R22 reprocessing, we will have to drop this 15 GeV cut from the AntiKt4LCTopo jet reconstruction (by setting jetFlags.useCalibJetThreshold to False), and instead apply a tau pt cut at the end of the tau reconstruction, in the tau thinning algorithm.

Cheers,
Bertrand

Bertrand Martin Dit Latour authored 4 years ago and Edward Moyse committed 4 years ago

Hello,

In R22, we will have a dedicated reconstruction at xAOD level targeting Long Lived Particles decaying to taus, which will use Large Radius Tracks within 0.4 of the tau axis, and we'll have dedicated LRT tunes for tau algorithms (tau ID, ...).
For the R22 reprocessing already, we plan to associated LRTs with taus, so that LRTs can be retrieved via tau->tracks(xAOD::TauJetParameters::LargeRadiusTrack), which will facilite a lot the navigation at xAOD level.
However, we don't want the LRTs to participate in the standard tau reconstruction, besides the tau-track association.
This MR implements a mechanism to skip Large Radius Tracks in the tau track classifier and all downstream tools, even if LRTs have been associated with taus in TauTrackFinder.

The modification is made in the tau track classifier. If the tool is instructed to ignore LRTs (via tauRecFlags), it will filter out the LRTs possibly associated with the tau, decorate these with default RNN track classification scores, and proceed to classify the non-LRTs. I also fixed a problem with an incorrectly set xAOD::TauJetParameters::unclassified tau track flag.

The RNN tau ID tool now explicitly ignores the tracks that are not classified, so it is not affected by the possible presence of LRTs associated with taus in TauTrackFinder.
I have checked that the RNN ID score is unchanged when we associate LRTs with taus but don't classify them.
More validation will follow.

In TauAlgorithmsHolder, deprecated tools were moved to the end of the file, and will be cleaned up at the next occasion.

Cheers,
Bertrand

Hello,

This MR deploys the tau thinning functionality (ATLTAU-1770) in RecExCommon_topOptions (at the moment, it is in the cell thinning block).
The thinning is controlled by a new tauRec pt flag, which is currently set 0 not to activate the thinning yet.
It will be set to ~10 GeV for the R22 reprocessing.

Cheers,
Bertrand

Bertrand Martin Dit Latour authored 4 years ago and Walter Lampl committed 4 years ago

Hello,

This MR is simply renaming CombinedP4FromRecoTaus to TauCombinedTES, as the former is not very harmonious and does not follow the tauRecTools tool naming convention.
It does not change the reconstruction output.

Cheers,
Bertrand

Hello,

This MR is adding tauRecFlags for the calibration files of the main tau algorithms, so that they can be changed on the fly with a preExec.
It will be very useful for the upcoming R22 tuning campaign, in case we need to rerun reconstruction with updated tunes.
A couple of deprecated flags were dropped. This MR does not change the reconstruction output.

Cheers,
Bertrand

Hello,

This MR is switching on the RNN track classification by default in the R22 tau reconstruction.
So far, we were using the R21-style set of 3 BDTs for tau track classification, but this won't be used in R22.
We were keeping it for the sake of having stable tau distributions in PHYSVAL and DQ checks.
Now that the 22.0.20 release is built for the November (final?) PHYSVAL check, we can switch this on.
It will be interesting to check the impact on SPOT plots (ATLASRECTS-5684), as many tau containers are only built for taus with 1-5 tracks (where the track multiplicity is that after track classification).

Cheers,
Bertrand

Added new flag to remove duplicate tau core tracks or not. Tidied a couple of other flags in TauRecBuilder

Bertrand Martin Dit Latour authored 4 years ago and Frank Winklmeier committed 4 years ago

Hello,

This MR is introducing a flag to skip tau candidates that have too many tau tracks.
The number of tracks corresponds to the number of tracks classified as good tau tracks by the tau track classifier.
Currently, we already run part of the tau reconstruction (tau substructure reconstruction) for candidates that have 1-5 tracks.
The new flag can be used to run the whole reconstruction only for taus with e.g. 0-5 tracks.
We would save some CPU and disk (relevant ticket: ATLASRECTS-5684).
The main motivation is that taus with >3 tracks will never be used in analyses.
0-track candidates should be kept as they are good candidates for e.g. neutral LLP->tau. The 1-3 track candidates are for physics. Keeping up to e.g. 5-track candidates gives some flexibility to retune and improve the track classifier at xAOD level, although that is somewhat unlikely to happen.

Cheers,
Bertrand

Guillermo Nicolas Hamity authored 4 years ago and Frank Winklmeier committed 4 years ago

LRT track container included in TauTrackFinder
Using InDetLargeD0TrackParticles as LRT container
name will make this happen

confusion

If useSubtractedCluster is false, we will use the inital cluster from
neutral PFOs oor charged PFOs. Otherwise, we will only use the charged
shower subtracted cluster from neutral PFOs.

Hello,

This MR is introducing a new tauRec flag to control the minimum pt of the seed jet, instead of a currently hardcoded 10 GeV cut.
This will be useful to study the impact of that cut on the tau reconstruction efficiency.
Some minor left-over pieces have been cleaned up.

Cheers,
Bertrand

Pass tau track container to track tools via execute function instead of a link from the tau. New functions created in base classes. Updated if statements in tau algorithms to call correct execute functions. Updated TauTrackRNN calibration file.

Adam Bailey authored 4 years ago and Adam Edward Barton committed 4 years ago

Updated TauSubstructureVariables and CaloClusterVariables to use new HelperFunction to navigate jet->clusters. Will now work with PFlow seeds

Updated function to get calo clusters from jets. Ignore repeat clusters, added flag to enable use of shower subtracted clusters. MvaTESVariableDecorator uses cluster instead of jet constituent for dR check"

Xiaozhong Huang authored 5 years ago and Edward Moyse committed 5 years ago

Two ART tests are added in this MR. One is q431, and the other is the
standalone tau reconstruction (ESD->AOD).
For each test, we will check the difference of root file and histograms
between the nightly build and reference.