Skip to content

Implementation of Clopper-Pearson confidence interval for t&p efficiencies

Federica Maria Simone requested to merge fsimone/spark_tnp:asym_errors into master

Addressing issue #5 (closed)

  • Clopper-Pearson confidence interval used to compute statistical uncertanties.
  • Asymmetric uncertanties are propagated to the SF computation and symmetrised by taking the maximum between up and down variations
  • For combining statistical uncertanties with systematics on the efficiencies, again the stat. are symmetrised by taking the maximum
  • Two additional histograms are stored for each variable/bin, where the up and down variations are assigned as uncertanties on the efficiencies.
  • The TGraphs have now asymmetric errors

Validation:

  1. Run the example commands from README.md/Quick Start to compute TightID/TrackerMuons SF on 2018 UL with Z
./tnp_fitter.py flatten muon generalTracks Z Run2018_UL configs/muon_example.json --baseDir ./example
./tnp_fitter.py fit muon generalTracks Z Run2018_UL configs/muon_example.json --baseDir ./example
./tnp_fitter.py prepare muon generalTracks Z Run2018_UL configs/muon_example.json --baseDir ./example
  1. Compare the results before and after modifying the code:
  • Impacts on SF uncertanties is very small

SF Before

SF_before

SF After

SF_after

  • The Asymmetric error bars on the efficiency plots are barely visible NUM_TightID_DEN_TrackerMuons_abseta_1_vs_pt

Values with old code:

root NUM_TightID_DEN_TrackerMuons_abseta_1_vs_pt.root
root [1] g_0_Data->Print()
x[0]=17.5, y[0]=0.958908, exl[0]=2.5, exh[0]=2.5, eyl[0]=0.000391978, eyh[0]=0.000391978
x[1]=22.5, y[1]=0.962418, exl[1]=2.5, exh[1]=2.5, eyl[1]=0.000212872, eyh[1]=0.000212872
x[2]=27.5, y[2]=0.964542, exl[2]=2.5, exh[2]=2.5, eyl[2]=0.000137995, eyh[2]=0.000137995
x[3]=35, y[3]=0.96584, exl[3]=5, exh[3]=5, eyl[3]=4.33261e-05, eyh[3]=4.33261e-05
x[4]=45, y[4]=0.966434, exl[4]=5, exh[4]=5, eyl[4]=6.26917e-05, eyh[4]=6.26917e-05
x[5]=55, y[5]=0.965741, exl[5]=5, exh[5]=5, eyl[5]=0.000140238, eyh[5]=0.000140238
x[6]=90, y[6]=0.964985, exl[6]=30, exh[6]=30, eyl[6]=0.000207744, eyh[6]=0.000207744

root [2] g_1_Simulation->Print()
x[0]=17.5, y[0]=0.966734, exl[0]=2.5, exh[0]=2.5, eyl[0]=0.000482083, eyh[0]=0.000482083
x[1]=22.5, y[1]=0.970592, exl[1]=2.5, exh[1]=2.5, eyl[1]=0.000308691, eyh[1]=0.000308691
x[2]=27.5, y[2]=0.972341, exl[2]=2.5, exh[2]=2.5, eyl[2]=0.000211466, eyh[2]=0.000211466
x[3]=35, y[3]=0.973864, exl[3]=5, exh[3]=5, eyl[3]=0.000101365, eyh[3]=0.000101365
x[4]=45, y[4]=0.97456, exl[4]=5, exh[4]=5, eyl[4]=9.24312e-05, eyh[4]=9.24312e-05
x[5]=55, y[5]=0.974206, exl[5]=5, exh[5]=5, eyl[5]=0.000210433, eyh[5]=0.000210433
x[6]=90, y[6]=0.974276, exl[6]=30, exh[6]=30, eyl[6]=0.000306223, eyh[6]=0.000306223

Values with new code:

root [3] g_0_Data->Print()
x[0]=17.5, y[0]=0.958908, exl[0]=2.5, exh[0]=2.5, eyl[0]=0.000307796, eyh[0]=0.000292204
x[1]=22.5, y[1]=0.962418, exl[1]=2.5, exh[1]=2.5, eyl[1]=0.000218431, eyh[1]=0.000181569
x[2]=27.5, y[2]=0.964542, exl[2]=2.5, exh[2]=2.5, eyl[2]=0.000142341, eyh[2]=0.000157659
x[3]=35, y[3]=0.96584, exl[3]=5, exh[3]=5, eyl[3]=3.95352e-05, eyh[3]=6.04648e-05
x[4]=45, y[4]=0.966434, exl[4]=5, exh[4]=5, eyl[4]=3.4279e-05, eyh[4]=6.5721e-05
x[5]=55, y[5]=0.965741, exl[5]=5, exh[5]=5, eyl[5]=0.000141161, eyh[5]=0.000158839
x[6]=90, y[6]=0.964985, exl[6]=30, exh[6]=30, eyl[6]=0.000184655, eyh[6]=0.000215345

root [4] g_1_Simulation->Print()
x[0]=17.5, y[0]=0.966734, exl[0]=2.5, exh[0]=2.5, eyl[0]=0.000533546, eyh[0]=0.000466454
x[1]=22.5, y[1]=0.970592, exl[1]=2.5, exh[1]=2.5, eyl[1]=0.000291836, eyh[1]=0.000308164
x[2]=27.5, y[2]=0.972341, exl[2]=2.5, exh[2]=2.5, eyl[2]=0.000241444, eyh[2]=0.000258556
x[3]=35, y[3]=0.973864, exl[3]=5, exh[3]=5, eyl[3]=6.41356e-05, eyh[3]=0.000135864
x[4]=45, y[4]=0.97456, exl[4]=5, exh[4]=5, eyl[4]=5.95668e-05, eyh[4]=0.000140433
x[5]=55, y[5]=0.974206, exl[5]=5, exh[5]=5, eyl[5]=0.000206341, eyh[5]=0.000193659
x[6]=90, y[6]=0.974276, exl[6]=30, exh[6]=30, eyl[6]=0.000275881, eyh[6]=0.000324119

Merge request reports