Creation of DaVinci test input files
We need a comprehensive record of how the input files for DaVinci tests are created. This has caused me (and @sesen) much pain the past days and I will not be responsible for this again.
An issue is obviously not the best place for this information but I put it here for now.
/eos/lhcb/wg/dpa/wp3/tests/hlt2_passthrough_thor_lines.dst
To create a file serving the purpose of For tests including option_davinci_tupling_DTF_pvs.py
HLT2 step
-
Creates HLT2 output which then goes through passthrough Sprucing for DaVinci test ...
-
Created using up-to-date
master
stack on 25/02/2024 using platformx86_64_v2-el9-gcc13+detdesc-opt
. (Note MC can only use detdesc platforms at this time) -
To run : inside
/Moore
./run gaudirun.py hlt2_bsjpsiphi_ppthorrealtime.py
hlt2_bsjpsiphi_ppthorrealtime.py
"""
Creates HLT2 output which then goes through passthrough Sprucing for DaVinci test ...
Created using up-to-date master stack on 25/02/2024 using platform `x86_64_v2-el9-gcc13+detdesc-opt`
(Note MC can only use detdesc platforms at this time)
To run : inside Moore
```./run gaudirun.py hlt2_bsjpsiphi_ppthorrealtime.py```
"""
from Moore import options, run_moore
from RecoConf.global_tools import stateProvider_with_simplified_geom, trackMasterExtrapolator_with_simplified_geom
from RecoConf.reconstruction_objects import reconstruction
from RecoConf.hlt2_global_reco import reconstruction as hlt2_reconstruction, make_fastest_reconstruction
from Hlt2Conf.lines import all_lines
import re
######## Following is for any regular pp thor job ########
# Taken from https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/options/hlt2_pp_thor.py
### explicitly remove lines that require hlt1_filter_code
def remove_lines(lines_dict, pattern_to_remove):
filtered = {
name: line
for name, line in lines_dict.items()
if re.search(pattern_to_remove, name) is None
}
return filtered
to_remove = [
'Hlt2IFT_SMOG2GECPassthrough', 'Hlt2IFT_SMOG2LumiPassthrough',
'Hlt2IFT_SMOG2MBPassthrough', 'Hlt2IFT_Femtoscopy_InclLambdaLL',
'Hlt2IFT_Femtoscopy_InclXiLLL', 'Hlt2IFT_Femtoscopy_InclOmegaLLL',
'Hlt2IFT_Femtoscopy_LambdaP', 'Hlt2IFT_Femtoscopy_LambdaP_lowK',
'Hlt2IFT_Femtoscopy_LambdaLambda', 'Hlt2IFT_Femtoscopy_LambdaLambda_lowK',
'Hlt2IFT_Femtoscopy_XiP', 'Hlt2IFT_Femtoscopy_XiP_lowK',
'Hlt2IFT_Femtoscopy_XiLambda', 'Hlt2IFT_Femtoscopy_XiLambda_lowK',
'Hlt2IFT_Femtoscopy_XiXi', 'Hlt2IFT_Femtoscopy_OmegaP',
'Hlt2IFT_Femtoscopy_OmegaP', 'Hlt2IFT_Femtoscopy_OmegaP_lowK',
'Hlt2IFT_Femtoscopy_OmegaLambda', 'Hlt2IFT_Femtoscopy_OmegaXi',
'Hlt2IFT_Femtoscopy_OmegaOmega', 'Hlt2QEE_DiElectronPrompt_PersistPhotons',
'Hlt2QEE_DiElectronPrompt_PersistPhotons_FULL',
'Hlt2QEE_DiElectronDisplaced_PersistPhotons',
'Hlt2QEE_DiElectronDisplaced_PersistPhotons_FULL',
'Hlt2QEE_DiElectronPrompt_PersistPhotonsSS',
'Hlt2QEE_DiElectronDisplaced_PersistPhotonsSS'
]
trunc_lines = all_lines
for remove in to_remove:
trunc_lines = remove_lines(trunc_lines, remove)
print("Removed lines: ", all_lines.keys() - trunc_lines.keys())
def make_lines():
return [builder() for builder in trunc_lines.values()]
public_tools = [
trackMasterExtrapolator_with_simplified_geom(),
stateProvider_with_simplified_geom()
]
######## Above is for any regular pp thor job ########
## Below is specific for the creation of DaVinci test input file ##
input_files = [
# Upgrade08 simulations of Bs2JpsiPhi avaliable in PRConfig at
# https://gitlab.cern.ch/lhcb-datapkg/PRConfig/-/blob/master/python/PRConfig/TestFileDB.py#L11029
'/eos/lhcb/grid/prod/lhcb/MC/Upgrade/XDIGI/00122720/0000/00122720_00000002_1.xdigi'
]
## Needed as these are 2020 MC samples
from RecoConf.decoders import default_ft_decoding_version
default_ft_decoding_version.global_bind(value=2)
from RecoConf.hlt1_muonid import make_muon_hits
make_muon_hits.global_bind(geometry_version=2)
options.input_files = input_files
options.input_type = 'ROOT'
options.input_raw_format = 0.5
options.evt_max = -1
options.simulation = True
options.data_type = 'Upgrade'
options.dddb_tag = 'dddb-20201211'
options.conddb_tag = 'sim-20201218-vc-mu100'
options.output_manifest_file = "hlt2_bsjpsiphi_ppthorrealtime.tck.json"
options.scheduler_legacy_mode = False
options.output_file = "hlt2_bsjpsiphi_ppthorrealtime.mdf"
options.output_type = "MDF"
with reconstruction.bind(from_file=False),\
hlt2_reconstruction.bind(make_reconstruction=make_fastest_reconstruction):
config = run_moore(options, make_lines, public_tools)
-
Log file here
/eos/lhcb/wg/dpa/wp3/tests/logs/hlt2_bsjpsiphi_ppthorrealtime.log
-
Have 145 events passing
Hlt2B2CC_BsToJpsiPhi_Detached
Hlt2 line
LAZY_AND: Hlt2B2CC_BsToJpsiPhi_DetachedDecisionWithOutput #=1000 Sum=145
- MDF output and json also at
/eos/lhcb/wg/dpa/wp3/tests/logs/
for now
Sprucing step
-
Creates Turbo (passthrough) Spruce output for DaVinci tests. Runs on top of the HLT2 file created above
-
Created using up-to-date
master
stack on 25/02/2024 using platformx86_64_v2-el9-gcc13+detdesc-opt
. -
Use
lbexec
to run : inside/Moore
./run lbexec Hlt2Conf.Sprucing_tests:spruce_passthrough spruce.yaml
where the yaml
is
spruce_bsjpsiphi_ppthorrealtime.yaml
process: Spruce
input_raw_format : 0.5
input_type : 'RAW'
input_files : [
'hlt2_bsjpsiphi_ppthorrealtime.mdf'
]
input_manifest_file : "hlt2_bsjpsiphi_ppthorrealtime.tck.json"
simulation : True
data_type : 'Upgrade'
conddb_tag: "sim-20201218-vc-mu100"
dddb_tag: "dddb-20201211"
output_file : 'spruce_bsjpsiphi_passthrough.{stream}.dst'
output_type : 'ROOT'
output_manifest_file : "spruce_bsjpsiphi_passthrough.tck.json"
input_process: Hlt2
-
Log file here
/eos/lhcb/wg/dpa/wp3/tests/logs/spruce_bsjpsiphi_ppthorrealtime.log
-
Note the output dst is called
spruce_bsjpsiphi_passthrough..dst
asstream
isNone
-
Inspecting the dst I see (using rawbanks from
/Event/default/RawEvent
)
>>> countevents_decision("Hlt2B2CC_BsToJpsiPhi_Detached")
Hlt2B2CC_BsToJpsiPhi_Detached fired 145 times
-
DST output and json also at
/eos/lhcb/wg/dpa/wp3/tests/logs/
for now -
This should be sufficient for the tests