Draft: Persist options to FileSummaryRecord (FSR)
Work on https://gitlab.cern.ch/lhcb-dpa/project/-/issues/271 .
Persist the options used to create a .dst file to its FileSummaryRecord. The goal is to have data files that are self contained i.e. information necessary for further processing should be contained in the file itself (dbtags etc.). Useful for Automatic configuration of Analysis productions for Run3 see DaVinci#145 .
Tested locally with
Moore/run lbexec Hlt2Conf.Sprucing_tests:spruce_test_lines test_spruce_data.yaml
And got this in the FSR:
... '8': '/Event/Spruce/HLT2/TISTOS/Hlt2Topo2Body/Particles', '9': '/Event/Spruce/HLT2/TISTOS/Hlt2Topo3Body/Particles'}, 'version': '0'}}}, 'LumiCounter.eventsByRun': {'counts': {'255620': 4818}, 'empty': False, 'type': 'LumiEventCounter'}, 'guid': '489064FE-00B4-11EF-9E3A-FA163E36415C', 'options': {'append_decoding_keys_to_output_manifest': True, 'auditors': [], 'buffer_events': 20000, 'compression': None, 'conddb_tag': None, 'conditions_version': 'master', 'control_flow_file': None, 'data_flow_file': None, 'data_type': 'Upgrade', 'dddb_tag': None, 'event_store': 'HiveWhiteBoard', 'event_timeout': None, 'evt_max': 5000, 'first_evt': 0, 'geometry_version': 'run3/trunk', 'histo_file': 'spruce_test_lines_histos.root', 'input_files': ['mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp1/data/2022_1_FULL_255620_00150001_0005.raw', 'mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp1/data/2022_1_FULL_255949_00150003_0000.raw'], 'input_manifest_file': None, 'input_process': 'Hlt2', 'input_raw_format': 0.5, 'input_type': 'RAW', 'memory_pool_size': 10485760, 'monitoring_file': None, 'msg_svc_format': '% F%35W%S %7W%R%T %0W%M', 'msg_svc_time_format': '%Y-%m-%d %H:%M:%S UTC', 'n_event_slots': 1, 'n_threads': 1, 'ntuple_file': None, 'output_file': 'spruce_test_lines_production.dst', 'output_level': 3, 'output_manifest_file': 'spruce_test_lines_production.tck.json', 'output_type': 'ROOT', 'persistreco_version': 0.0, 'phoenix_filename': None, 'preamble_algs': [], 'print_freq': 10000, 'process': 'Spruce', 'python_logging_level': 20, 'require_specific_decoding_keys': [], 'scheduler_legacy_mode': True, 'simulation': False, 'tck': None, 'use_iosvc': False, 'write_decoding_keys_to_git': True, 'xml_file_catalog': None, 'xml_summary_file': None}
This is the yaml with the options used for the local test:
data_type: 'Upgrade'
simulation: False
input_raw_format: 0.5
geometry_version: run3/trunk
conditions_version: master
process: Spruce
input_type : 'RAW'
input_files : [
'mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp1/data/2022_1_FULL_255620_00150001_0005.raw',
'mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp1/data/2022_1_FULL_255949_00150003_0000.raw'
]
evt_max : 5000
persistreco_version: 0.0
output_type : 'ROOT'
input_process: Hlt2
histo_file : 'spruce_test_lines_histos.root'
output_file : 'spruce_test_lines_production.dst'
output_manifest_file : 'spruce_test_lines_production.tck.json'