Add HS Vertex Reco/Selection Efficiency and Reso Plots to IDPVM
Attempts to add Figures 8-12 from ATL-PHYS-PUB-2019-015. The scheme for each is:
- Selection efficiency: determine the "best" reco HS vertex using the sumpt2 of reco tracks and using the sum of truth HS track weights (
InDetVertexTruthMatchUtils::bestHardScatterMatch?) and see if they match (calculate local PU density using the truth vertex for the sumpt2 best reco HS vertex) - Reconstruction efficiency: find the truth vertex for every reco vertex and check if it is present in our vector of truth HS vertices for the event - if so, we count this as a "pass"
- HS resolutions: for each vertex in our vector of truth HS vertices (I think we only ever expect 1), calculate the "truth" resolution as the absolute difference between that truth vertex and its matched reco vertex longitudinally and transversely (this is what's in the note) as well as the "reco" resolution using the covariance matrix and error propagation (which I hope I did correctly...)
Additionally, I have also fixed a seg fault which shows up when trying to fill histograms in InDetPerfPlot_VertexTruthMatching.cxx when m_iDetailLevel >= 200 by adding iDetailLevel to the constructor of InDetRttPlots.
Merge request reports
Activity
Resolved by Matthew Joseph BassoI have a few other points: I noticed in !32441 (merged) that the default value for
m_pileupSwitchhas been set to "HardScatter", which I set to "All" in my MR as I need to get a vector of truth HS and PU vertices for the local PU density calculation. But let me know if we want "HardScatter" as the default and I'll make a workaround.Additionally, something that was unintuitive to me was that the y values in the xml config for the IDPVM plots control the acceptance of the y range and not its limits on the graph (at least for TProfile): is this expected?
Finally, the sample I've been using for testing, /afs/cern.ch/user/n/npetters/workDir/public/forIgor/tmp.AOD, seems to always return false for
m_truthEventName.key().empty()inInDetPhysValMonitoringTool.cxx, so I haven't been able to test anything. It's not obvious to me if this has been the case for my tests in previous MRs. Is this expected or is there some other switch I'm supposed to turn on? !32544 (merged) seems sort of related, but it doesn't change the function I've been using (InDetPhysValMonitoringTool::getTruthVertices).Edited by Matthew Joseph Basso- Last reply by Matthew Joseph Basso
added InnerDetector master review-pending-level-1 labels
Athena AthSimulation AnalysisBase AthGeneration externals cmake make required tests optional tests Full details available on this CI monitor view
added 6 commits
- ea1cf2ff - Speedup local PU density calc
- a20dc712 - C-style cast to Cpp-style cast
- b6fcaba7 - Replace inInterable with std::find
- 007ba17c - Use auto b/c idk what the iterator's type is
- d988262a - Don't explicitly cast ints to floats, TProfile::Fill does this for us
- ae0a591f - Protect against NULL in getTruthVertex
Toggle commit list-
Athena AthSimulation AnalysisBase AthGeneration externals cmake make required tests 
optional tests Full details available on this CI monitor view
-
I made one other tiny update to the truth event/vertex container retrieval in IDPVMTool: I now load the truth event (HS) container as well as the truth particle container when m_pileupSwitch is All or HardScatter (not just HardScatter). I hope this is OK.
Also, InDetPerfPlot_VertexTruthMatching::getTruthVertex prints a warning, "TruthEventMatchingInfos DECORATOR yields empty vector -- returning nullptr!, for the last vertex of the container for every single event. I don't know if this is explainable.
Finally, here are some plots from running the tool with m_pileupSwitch == All, you can see that the reco values for resos are quite flat in PU and generally smaller than the truth (|reco-truth|) resos, but they're of the same scale. They are also similar-ish to those in the PUB note. The vertex selection and reco efficiencies are nearly 1, but maybe this is an effect of stats. I assume for the reco efficiency, we just want to check that any reco vertex (not necessarily the best one via sumpt2 or the truth pkg, I assume to disentangle this from the selection efficiency) is matched to the truth HS vertex to call it a pass. But let me know if anything looks wrong.
-
Athena AthSimulation AnalysisBase AthGeneration externals cmake make required tests optional tests Full details available on this CI monitor view
Resolved by Matthew Joseph BassoLooks good to me, it was already extensively reviewed and the test failure is not related to this MR, approving --L1
- Last reply by Frank Winklmeier
added review-approved label and removed review-pending-level-1 label
added review-user-action-required label and removed review-approved label
-
@npetters @kostyuk @glee I've attached updated plots (updated_plots.pdf). Now the reco efficiency is determined by whether the best reco HS vertex as determined by the truth pkg is within 0.1 mm of the truth HS vertex AND if there are no other reco vertices closer than the best one. Also, I now determine the truth resolutions by fitting a gaussian to |truth - reco| (I understand that there is a class,
ResolutionHelperfor producing these plots, but it was easier for me to implement a custom function from Graham for now). For the first bin of the longitudinal resolution plot, the fit does not converge properly (I think...) and so the result is set to 0 (I have confirmed that there are the same number of raw events in the y profile, however). I think this might be an issue of stats. The binning in the y direction (|truth - reco|) may have to be updated for these plots sometime in the future. Additionally, because I useSetBinContenton these resolution plots, I lose the ability to track raw counts per bin (I assume the sum of raw events in the y profile). Maybe there's a better way to handle this.Let me know if anything looks wrong and if the code changes look OK.
Edited by Matthew Joseph Basso 
Resolved by Matthew Joseph BassoThanks @mbasso. Most of the changes look reasonable to me, but I would like somebody else who was present at the meeting to take a look as well. I have a couple of questions on the resolution though.
- If you fit the abs with a Gaussian, you will most likely only end up worsening the errors on the results, i.e. you are basically cutting out half of the degrees of freedom. Can you try without the abs and check the fit results to see how they change?
- I am not sure this is a good idea to really check if a fit failed. If you run a simple local test you will see that is very likely that you get status = 0 also when the fit is declared as failed. Also, the check is on (%1000) but you are not calling the "improve" option, i.e. you do not use "M" when you call the fit, so according to the equation (fitStatus = migradResult + 10 * minosResult + 100 * hesseResult + 1000 * improveResult) you will never get the last part. To understand what really makes your first bin empty, can you please plot the fit results as well as the 2D histograms?
Edited by Valentina Cairo- Last reply by Matthew Joseph Basso
added 6 commits
- 233bdc8c - Remove ResolutionHelper (for now?)
- 0b750656 - Remove abs on longitudinal reso (not much we can do for transverse?)
- 5cac37c4 - Remove status() check on fit result
- e4ff5f0b - Book 2D hists for trans/long reso as default
- 3ef5b9e6 - Remove extra status check
- a95da310 - Bin 2D long reso - -> + too
Toggle commit listadded review-pending-level-1 label and removed review-user-action-required label
-
Athena AthSimulation AnalysisBase AthGeneration externals cmake make required tests optional tests Full details available on this CI monitor view
Resolved by Matthew Joseph Basso- Last reply by Matthew Joseph Basso
added review-pending-level-2 label and removed review-pending-level-1 label
added review-approved label and removed review-pending-level-2 label
mentioned in commit 9919e87f
added sweep:ignore label
