Unifying how Run 3 applications are run
This is a proposal for how to unify the running of Run 3 applications across both interactive and grid style running.
Current usages of
davinci run-mc would be replaced by a new command provided by the
This is driven by the needs of grid jobs as well as avoiding fragmentation between how applications are ran.
There are a few constraints which have lead me to this idea:
Provide a reliable interface for setting the configuration needed running large numbers of jobs (definitely
The common part in
LHCbshould be minimal and leave flexibility for each application while still ensuring some configuration can be set from outside
One and only one solution for all applications in LHCb (Moore/Allen/DaVinci/Gauss, interactively/Ganga/DIRAC/LHCbPR)
The arguments are:
- First: Qualifier of the module to import and function to call
- Second: YAML files which are used to populate
$GAUDIAPPNAME.Options. Multiple YAML files can be provided here which will be overlaid before loading and LHCbDIRAC will provide a command for generating the basic options from the bookkeeping.
- Remaining (optional): Arguments which are passed to the function as strings
The user provided function (argument 1) would always take an
options argument that replaces the current global (
Options class would be inherited from
DaVinci extending it as needed.
All properties on the
options object would be loaded from the YAML and the object would then be immutable.
This function would return an instance of
PyConf.application.ComponentConfig, which would typically be generated a function like
I propose renaming this function to something like
make_config to avoid confusion about who is actually starting the event loop.
lbexec AppConfig.FEST.Sprucing:passthrough options.yaml lbexec AppConfig.FEST.Sprucing:exclusive options.yaml lbexec my_options:main options.yaml+data.yaml "DpToKee_SS"
And the corresponding function signatures (the type annotations aren't required):
from PyConf.application import ComponentConfig from Moore import Options, run_moore from RecoConf.reconstruction_objects import reconstruction def passthrough(options: Options) -> ComponentConfig: with reconstruction.bind(from_file=True, spruce=True): return run_moore(options, ...) def exclusive(options: Options) -> ComponentConfig: with reconstruction.bind(from_file=True, spruce=True): return run_moore(options, ...) from DaVinci import Options, generate_config def main(options: Options, decay: str) -> ComponentConfig: return generate_config(configured_FunTuple(...))
I have a (very) basic prototype working locally for Moore and DaVinci and I've pushed the main parts here: