Erroneous behaviour in lines that use same hlt_filters
Description
If 2 lines share the same hlt{1,2}_filter the "second" instance of the filter does not perform any selection.
In the minimal reproducer Hlt2test_1 and Hlt2test_2 lines are identical apart from their name. Both have
hlt1_filter_code=[
"Hlt1TrackMVADecision", "Hlt1TwoTrackMVADecision",
"Hlt1LowPtDiMuonDecision"
],However the log shows that the filter for Hlt2test_2 does not do anything
LAZY_AND: Hlt2test_1_DecisionWithOutput #=10000 Sum=3 Eff=|(0.03000000 +- 0.0173179)%|
LAZY_AND: Hlt2test_1_ #=10000 Sum=3 Eff=|(0.03000000 +- 0.0173179)%|
DeterministicPrescaler/Hlt2test_1__Prescaler #=10000 Sum=10000 Eff=|( 100.0000 +- 0.00000 )%|
VoidFilter/Hlt2test_1__Hlt1Filter #=10000 Sum=6174 Eff=|( 61.74000 +- 0.486022)%|LAZY_AND: Hlt2test_2_DecisionWithOutput #=10000 Sum=7 Eff=|(0.07000000 +- 0.0264483)%|
LAZY_AND: Hlt2test_2_ #=10000 Sum=7 Eff=|(0.07000000 +- 0.0264483)%|
DeterministicPrescaler/Hlt2test_2__Prescaler #=10000 Sum=10000 Eff=|( 100.0000 +- 0.00000 )%|
VoidFilter/Hlt2test_2__Hlt1Filter #=10000 Sum=10000 Eff=|( 100.0000 +- 0.00000 )%|
If I change the filter to
hlt1_filter_code=["Hlt1TrackMVADecision", "Hlt1TwoTrackMVADecision"],for Hlt2test_2 only, then things work again
LAZY_AND: Hlt2test_1_DecisionWithOutput #=10000 Sum=3 Eff=|(0.03000000 +- 0.0173179)%|
LAZY_AND: Hlt2test_1_ #=10000 Sum=3 Eff=|(0.03000000 +- 0.0173179)%|
DeterministicPrescaler/Hlt2test_1__Prescaler #=10000 Sum=10000 Eff=|( 100.0000 +- 0.00000 )%|
VoidFilter/Hlt2test_1__Hlt1Filter #=10000 Sum=6174 Eff=|( 61.74000 +- 0.486022)%|LAZY_AND: Hlt2test_2_DecisionWithOutput #=10000 Sum=1 Eff=|(0.01000000 +- 0.00999950)%|
LAZY_AND: Hlt2test_2_ #=10000 Sum=1 Eff=|(0.01000000 +- 0.00999950)%|
DeterministicPrescaler/Hlt2test_2__Prescaler #=10000 Sum=10000 Eff=|( 100.0000 +- 0.00000 )%|
VoidFilter/Hlt2test_2__Hlt1Filter #=10000 Sum=3183 Eff=|( 31.83000 +- 0.465817)%|To replicate
Minimal reproducer is a copy of the options used for the hlt2_pp_thor_data_2022.qmt test. I have changed the lines running only
"""
This is a copy of the options used for the hlt2_pp_thor_data_2022.qmt test. I have changed the lines running only
"""
from Moore import options, run_moore
from RecoConf.reconstruction_objects import reconstruction
from RecoConf.hlt2_global_reco import reconstruction as hlt2_reconstruction, make_light_reco_pr_kf_without_UT
from Hlt2Conf.lines.charm.d0_to_hh import make_charm_kaons, make_dzeros
from Hlt2Conf.lines.charm.prefilters import charm_prefilters
from Moore.lines import Hlt2Line
from RecoConf.hlt2_tracking import (
make_PrKalmanFilter_noUT_tracks, make_PrKalmanFilter_Seed_tracks,
make_PrKalmanFilter_Velo_tracks, make_TrackBestTrackCreator_tracks,
get_UpgradeGhostId_tool_no_UT)
from PyConf.Algorithms import VeloRetinaClusterTrackingSIMD, VPRetinaFullClusterDecoder
from RecoConf.hlt1_tracking import (
make_VeloClusterTrackingSIMD, make_RetinaClusters,
get_global_measurement_provider, make_velo_full_clusters)
from RecoConf.hlt1_muonid import make_muon_hits
from RecoConf.calorimeter_reconstruction import make_digits
options.histo_file = "histos_hlt2_pp_thor_data_2022.root"
options.output_file = "data_hlt2_pp_thor_data_2022.mdf"
options.output_type = "MDF"
options.event_store = 'EvtStoreSvc'
options.scheduler_legacy_mode = False
options.evt_max = 10000
##Define arbitrary line
def test1_line(name='Hlt2test_1_', prescale=1):
kaons = make_charm_kaons()
dzeros = make_dzeros(kaons, kaons, 'D0 -> K- K+')
return Hlt2Line(
name=name,
algs=charm_prefilters() + [dzeros],
hlt1_filter_code=[
"Hlt1TrackMVADecision", "Hlt1TwoTrackMVADecision",
"Hlt1LowPtDiMuonDecision"
],
prescale=prescale)
#Define the line again with a different name
def test2_line(name='Hlt2test_2_', prescale=1):
kaons = make_charm_kaons()
dzeros = make_dzeros(kaons, kaons, 'D0 -> K- K+')
return Hlt2Line(
name=name,
algs=charm_prefilters() + [dzeros],
hlt1_filter_code=[
"Hlt1TrackMVADecision", "Hlt1TwoTrackMVADecision",
"Hlt1LowPtDiMuonDecision"
],
prescale=prescale)
def make_lines():
return [test1_line(), test2_line()]
with reconstruction.bind(from_file=False),\
make_light_reco_pr_kf_without_UT.bind(skipRich=False, skipCalo=False, skipMuon=False),\
make_TrackBestTrackCreator_tracks.bind(max_chi2ndof=8.0),\
make_PrKalmanFilter_Velo_tracks.bind(max_chi2ndof=8.0),\
make_PrKalmanFilter_noUT_tracks.bind(max_chi2ndof=8.0),\
make_PrKalmanFilter_Seed_tracks.bind(max_chi2ndof=8.0),\
make_VeloClusterTrackingSIMD.bind(algorithm=VeloRetinaClusterTrackingSIMD),\
get_UpgradeGhostId_tool_no_UT.bind(velo_hits=make_RetinaClusters),\
make_muon_hits.bind(geometry_version=3),\
make_digits.bind(calo_raw_bank=True),\
make_velo_full_clusters.bind(make_full_cluster=VPRetinaFullClusterDecoder),\
get_global_measurement_provider.bind(velo_hits=make_RetinaClusters),\
hlt2_reconstruction.bind(make_reconstruction=make_light_reco_pr_kf_without_UT):
config = run_moore(options, make_lines, public_tools=[])Run with
lb-run -c best Moore/v54r10 gaudirun.py '$MOOREROOT/tests/options/mdf_input_and_conds_data_2022.py' hlt2_example_2022.pyExtra details
I discovered this whilst trying to create passthrough lines for the TURCAL stream where the requirement is that
- 10% of the data needs to keep all DET RawBanks
To do this I created 2 passthrough lines selecting on the same hlt2 decisions, but one has a different rawbank persistency and a prescale. These passthrough lines would then be different output streams. So the problem is NOT specific to HLT1.
Might be connected to allow_duplicate_instances_with_distinct_names used in the filters https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/lines.py#L335?
Could also be behind #605 (closed) (cc @abertoli @shunan @matzeni) ?
Edited by Nicole Skidmore