Add new QEE HLT2 and Sprucing Lines to master

FYI @wbarter @lcalefic @ncooke @jideng @tmombach @fredi @lsestini @ausachov @cvazquez @hyin @dzuliani (tagging conveners, liaisons and those who contributed MRs to this effort, apologies if I missed anyone!)

This MR adds the following QEE-specific developments to master:

• Enables persistreco for all the QEE dilepton lines (!933 (merged))
• Adds an example Z->MuMu sprucing line (!853 (merged))
• Adds a copy of the Hlt2SingleHighPtMuonLine with no MuonID, appropriately prescaled (see !1000 (merged)),
• Modifies our di-jet HLT2 lines from 1->18 lines with differing thresholds and tags (!1001 (merged)),
• Adds a suite of HLT2 single high-pT electron lines (!1004 (merged)),
• Adds HLT2 lines for Z->ee and same-sign high-mass dielectron candidates (!855 (merged)),
• Adds sprucing lines for ttbar->electron+muon, and boson+jet (!1005 (merged) and !1321 (merged)),
• Migrates the HLT2 Z->MuMu lines to ThOr (!1374 (merged)),
• Adds diphoton HLT2 lines (!1171 (merged)),
• Prefer Hlt2Line over HltLine (!1355 (merged)) and book our lines separately for HLT2 and sprucing (!1386 (merged)),
• Adds di- and tri-jet sprucing lines (!1295 (merged)).

It also adds QEE-specific example options files, which can be removed and kept on a QEE-specific branch to avoid cluttering master if desired.

Most of the lines are not migrated to ThOr due to missing functors, which are all known and documented in various issues. All lines here that can be migrated at this time have been migrated.

Rates and efficiency information will be collated and posted in the description.

added qee label

assigned to @rjhunter

added RTA label

changed the description

Efficiences of the single- and di-muon HLT2 lines on 1k Z\to\mu\mu with both muons reconstructible, 2 < \eta < 4.5 and p_T > 20 GeV:

Integrated HLT efficiencies for the lines with denominator: HighTruthPtMuon
------------------------------------------------------------------------------------
Line:	 Hlt2MuMuSSLineDecision                            	 Efficiency:	 0.016 +/- 0.008
Line:	 Hlt2SingleHighPtMuonIsoLineDecision               	 Efficiency:	 0.972 +/- 0.010
Line:	 Hlt2SingleHighPtMuonLineDecision                  	 Efficiency:	 1.000 +/- 0.000
Line:	 Hlt2SingleHighPtMuonNoMuIDLineDecision            	 Efficiency:	 0.083 +/- 0.017
Line:	 Hlt2SingleHighPtMuonPrescaleLineDecision          	 Efficiency:	 0.056 +/- 0.014
Line:	 Hlt2ZToMuMuLineDecision                           	 Efficiency:	 0.921 +/- 0.017
Line:	 Z0_Hlt2MuMuSSLineDecisionTrueSim                  	 Efficiency:	 0.000 +/- 0.000
Line:	 Z0_Hlt2SingleHighPtMuonIsoLineDecisionTrueSim     	 Efficiency:	 0.972 +/- 0.010
Line:	 Z0_Hlt2SingleHighPtMuonLineDecisionTrueSim        	 Efficiency:	 1.000 +/- 0.000
Line:	 Z0_Hlt2SingleHighPtMuonNoMuIDLineDecisionTrueSim  	 Efficiency:	 0.083 +/- 0.017
Line:	 Z0_Hlt2SingleHighPtMuonPrescaleLineDecisionTrueSim	 Efficiency:	 0.056 +/- 0.014
Line:	 Z0_Hlt2ZToMuMuLineDecisionTrueSim                 	 Efficiency:	 0.921 +/- 0.017
------------------------------------------------------------------------------------
Finished printing integrated HLT efficiencies for denominator: HighTruthPtMuon
------------------------------------------------------------------------------------

di-jet lines: efficiencies on 1000 b\bar{b} events, and rates on 10k min. bias events, as reported by @ncooke:

Line	Prescale	Rate / kHz	Efficiency (bbbar events) (%)
Hlt2DiLightJet10GeVLine	0.001	0.1 +/- 0.09	0.000000 +- 0.00000
Hlt2DiLightJet15GeVLine	0.01	0.2 +/- 0.14	0.7000000 +- 0.263647
Hlt2DiLightJet20GeVLine	0.05	0.1 +/- 0.09	1.600000 +- 0.396787
Hlt2DiLightJet25GeVLine	0.25	0.1 +/- 0.09	5.000000 +- 0.689202
Hlt2DiLightJet30GeVLine	0.5	0.1 +/- 0.09	6.000000 +- 0.750999
Hlt2DiLightJet35GeVLine	1.0	0.1 +/- 0.09	7.100000 +- 0.812151
Hlt2DiTopoTagJet10GeVLine	0.005	0.3 +/- 0.17	0.2000000 +- 0.141280
Hlt2DiTopoTagJet15GeVLine	0.01	0.2 +/- 0.14	0.2000000 +- 0.141280
Hlt2DiTopoTagJet20GeVLine	0.05	0.1 +/- 0.09	0.9000000 +- 0.298647
Hlt2DiTopoTagJet25GeVLine	0.5	0.3 +/- 0.17	11.20000 +- 0.997276
Hlt2DiTopoTagJet30GeVLine	0.5	0.0 +/- 0.0	5.900000 +- 0.745111
Hlt2DiTopoTagJet35GeVLine	1.0	0.1 +/- 0.09	6.700000 +- 0.790639
Hlt2DiSVTagJet10GeVLine	0.025	0.1 +/- 0.09	0.7000000 +- 0.263647
Hlt2DiSVTagJet15GeVLine	0.1	0.0 +/- 0.0	2.000000 +- 0.442719
Hlt2DiSVTagJet20GeVLine	1.0	0.0 +/- 0.0	21.30000 +- 1.29472
Hlt2DiSVTagJet25GeVLine	1.0	0.0 +/- 0.0	16.00000 +- 1.15931
Hlt2DiSVTagJet30GeVLine	1.0	0.0 +/- 0.0	9.200000 +- 0.913980
Hlt2DiSVTagJet35GeVLine	1.0	0.0 +/- 0.0	5.500000 +- 0.720937

where the efficiency denominator is all the events that were run over.

added 5 commits

• b48d746d - Persist the HLT2 configuration and MDF for the jet sprucing lines
• 263a581f - Update persistreco example
• aacb0a57 - Update the sprucing example
• b5bc57ed - Better name for sprucing example
• 20f17eb2 - Add bbbar sprucing example

Compare with previous version

added 1 commit

• d02279b5 - Fix formatting

Compare with previous version

added 2 commits

• 4169506b - Jets sprucing lines
• 04a34747 - Merge branch 'sprucing_jets' into 'qee_upgrade'

Compare with previous version

changed the description

@poluekt @mvesteri how would you like this to be reviewed/how would you like us to proceed towards getting this merged?

I have some rates pending, and I'll also show the efficiencies of the electron lines on Z\to ee shortly.

marked this merge request as ready

Rates on 50k HLT1-filtered minimum bias

HLT rates: 
----------------------------------------------------------------------------------------------------
Line:	 Hlt2DiLightJet10GeVLineDecision             	 Incl: 0.16 +/- 0.05 kHz, Excl: 0.16 +/- 0.05 kHz
Line:	 Hlt2DiLightJet15GeVLineDecision             	 Incl: 0.1  +/- 0.04 kHz, Excl: 0.1  +/- 0.04 kHz
Line:	 Hlt2DiLightJet20GeVLineDecision             	 Incl: 0.12 +/- 0.04 kHz, Excl: 0.08 +/- 0.03 kHz
Line:	 Hlt2DiLightJet25GeVLineDecision             	 Incl: 0.06 +/- 0.03 kHz, Excl: 0.02 +/- 0.01 kHz
Line:	 Hlt2DiLightJet30GeVLineDecision             	 Incl: 0.14 +/- 0.05 kHz, Excl: 0.02 +/- 0.01 kHz
Line:	 Hlt2DiLightJet35GeVLineDecision             	 Incl: 0.2  +/- 0.06 kHz, Excl: 0.04 +/- 0.02 kHz
Line:	 Hlt2DiSVTagJet10GeVLineDecision             	 Incl: 0.02 +/- 0.01 kHz, Excl: 0.02 +/- 0.01 kHz
Line:	 Hlt2DiSVTagJet15GeVLineDecision             	 Incl: 0.08 +/- 0.03 kHz, Excl: 0.04 +/- 0.02 kHz
Line:	 Hlt2DiSVTagJet20GeVLineDecision             	 Incl: 0.2  +/- 0.06 kHz, Excl: 0.12 +/- 0.04 kHz
Line:	 Hlt2DiSVTagJet25GeVLineDecision             	 Incl: 0.04 +/- 0.02 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2DiSVTagJet30GeVLineDecision             	 Incl: 0.02 +/- 0.01 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2DiSVTagJet35GeVLineDecision             	 Incl: 0.02 +/- 0.01 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2DiTopoTagJet10GeVLineDecision           	 Incl: 0.24 +/- 0.06 kHz, Excl: 0.24 +/- 0.06 kHz
Line:	 Hlt2DiTopoTagJet15GeVLineDecision           	 Incl: 0.04 +/- 0.02 kHz, Excl: 0.04 +/- 0.02 kHz
Line:	 Hlt2DiTopoTagJet20GeVLineDecision           	 Incl: 0.08 +/- 0.03 kHz, Excl: 0.06 +/- 0.03 kHz
Line:	 Hlt2DiTopoTagJet25GeVLineDecision           	 Incl: 0.14 +/- 0.05 kHz, Excl: 0.06 +/- 0.03 kHz
Line:	 Hlt2DiTopoTagJet30GeVLineDecision           	 Incl: 0.08 +/- 0.03 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2DiTopoTagJet35GeVLineDecision           	 Incl: 0.08 +/- 0.03 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2EESSLineDecision                        	 Incl: 0.08 +/- 0.03 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2MuMuSSLineDecision                      	 Incl: 0.0  +/- 0.0  kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2SingleHighPtElectronIsoLineDecision     	 Incl: 0.84 +/- 0.12 kHz, Excl: 0.34 +/- 0.08 kHz
Line:	 Hlt2SingleHighPtElectronLineDecision        	 Incl: 0.62 +/- 0.11 kHz, Excl: 0.1  +/- 0.04 kHz
Line:	 Hlt2SingleHighPtElectronPrescaleLineDecision	 Incl: 0.26 +/- 0.07 kHz, Excl: 0.22 +/- 0.06 kHz
Line:	 Hlt2SingleHighPtMuonIsoLineDecision         	 Incl: 0.12 +/- 0.04 kHz, Excl: 0.08 +/- 0.03 kHz
Line:	 Hlt2SingleHighPtMuonLineDecision            	 Incl: 0.04 +/- 0.02 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2SingleHighPtMuonNoMuIDLineDecision      	 Incl: 0.36 +/- 0.08 kHz, Excl: 0.22 +/- 0.06 kHz
Line:	 Hlt2SingleHighPtMuonPrescaleLineDecision    	 Incl: 0.02 +/- 0.01 kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2SingleVHighPtElectronLineDecision       	 Incl: 0.9  +/- 0.13 kHz, Excl: 0.72 +/- 0.11 kHz
Line:	 Hlt2ZToEELineDecision                       	 Incl: 0.04 +/- 0.02 kHz, Excl: 0.02 +/- 0.01 kHz
Line:	 Hlt2ZToMuMuLineDecision                     	 Incl: 0.0  +/- 0.0  kHz, Excl: 0.0  +/- 0.0  kHz
Line:	 Hlt2_ALPsToGammaGamma_LineDecision          	 Incl: 0.48 +/- 0.09 kHz, Excl: 0.44 +/- 0.09 kHz
----------------------------------------------------------------------------------------------------
Hlt2 Total:                                              Rate:	 4 +/- 0.3 kHz
----------------------------------------------------------------------------------------------------

You'll notice that the rate for Hlt2ZToMuMuLineDecision and Hlt2MuMuSSLineDecision is zero, because the expected rate for these lines is too low to be seen with 50k events. When added to qee_upgrade it was tested that, if we loosen the thresholds on these lines, they do indeed work.

Electron lines: efficiencies on 900 Z \to ee events, with both electrons reconstructible, 2 < \eta < 4.5 and p_T > 20 GeV:

Integrated HLT efficiencies for the lines with denominator: HighTruthPtMuon
------------------------------------------------------------------------------------
Line:	 Hlt2EESSLineDecision                                  	 Efficiency:	 0.100 +/- 0.019
Line:	 Hlt2SingleHighPtElectronIsoLineDecision               	 Efficiency:	 0.883 +/- 0.021
Line:	 Hlt2SingleHighPtElectronLineDecision                  	 Efficiency:	 0.917 +/- 0.018
Line:	 Hlt2SingleHighPtElectronPrescaleLineDecision          	 Efficiency:	 0.050 +/- 0.014
Line:	 Hlt2SingleVHighPtElectronLineDecision                 	 Efficiency:	 0.621 +/- 0.031
Line:	 Hlt2ZToEELineDecision                                 	 Efficiency:	 0.683 +/- 0.030
Line:	 Z0_Hlt2EESSLineDecisionTrueSim                        	 Efficiency:	 0.000 +/- 0.000
Line:	 Z0_Hlt2SingleHighPtElectronIsoLineDecisionTrueSim     	 Efficiency:	 0.871 +/- 0.022
Line:	 Z0_Hlt2SingleHighPtElectronLineDecisionTrueSim        	 Efficiency:	 0.896 +/- 0.020
Line:	 Z0_Hlt2SingleHighPtElectronPrescaleLineDecisionTrueSim	 Efficiency:	 0.050 +/- 0.014
Line:	 Z0_Hlt2SingleVHighPtElectronLineDecisionTrueSim       	 Efficiency:	 0.588 +/- 0.032
Line:	 Z0_Hlt2ZToEELineDecisionTrueSim                       	 Efficiency:	 0.633 +/- 0.031
------------------------------------------------------------------------------------
Finished printing integrated HLT efficiencies for denominator: HighTruthPtMuon
------------------------------------------------------------------------------------

added RTA WP3 label

Hi @rjhunter , can this be tested or waiting for some additional development?

added ci-test-triggered label

• [2022-03-07 15:10] Validation started with lhcb-master-mr#3885