DaVinci merge requestshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests2022-11-24T09:49:22+01:00https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/731Make "process" a required input and configure pyconf functions globally once ...2022-11-24T09:49:22+01:00Abhijit MathadMake "process" a required input and configure pyconf functions globally once with user inputCloses https://gitlab.cern.ch/lhcb/DaVinci/-/issues/59 and closes https://gitlab.cern.ch/lhcb/DaVinci/-/issues/74.
- [x] Make process a required input.
- [x] Make "options.process" as the default input to algorithms (e.g. truth matching...Closes https://gitlab.cern.ch/lhcb/DaVinci/-/issues/59 and closes https://gitlab.cern.ch/lhcb/DaVinci/-/issues/74.
- [x] Make process a required input.
- [x] Make "options.process" as the default input to algorithms (e.g. truth matching).
- [x] Make process a required input to truth-matching algorithm (related to issue: https://gitlab.cern.ch/lhcb/DaVinci/-/issues/62) and also set correct root-in-tes for `Turbo` process.
To be tested with: https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3806, https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/1812 and https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/914Chris BurrAbhijit MathadDavide FazziniChris Burrhttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/564Lint/clean-up/fix new configuration2021-07-16T10:05:38+02:00Eduardo RodriguesLint/clean-up/fix new configurationFYI @dfazzini and @mamartin.
Relates to DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/137.FYI @dfazzini and @mamartin.
Relates to DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/137.Patrick KoppenburgPatrick Koppenburghttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/514Integration of PyConf in the new DaVinci configuration2021-04-21T16:18:03+02:00Davide FazziniIntegration of PyConf in the new DaVinci configurationIntegrate the PyConf control flow in the new DaVinci configuration.
Work related to DPA's master task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/48.Integrate the PyConf control flow in the new DaVinci configuration.
Work related to DPA's master task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/48.Davide FazziniDavide Fazzinihttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/706Initial lbexec support2022-06-10T10:07:12+02:00Chris BurrInitial lbexec supportWhen reviewing this merge request the individual commits likely make the most sense. In summary:
* 6843d61be8d4c9a8fe6172ce9a72764cc93c4394 Adds the two needed parts for lbexec, more specifically:
* A `DaVinci.Options` class which is ...When reviewing this merge request the individual commits likely make the most sense. In summary:
* 6843d61be8d4c9a8fe6172ce9a72764cc93c4394 Adds the two needed parts for lbexec, more specifically:
* A `DaVinci.Options` class which is used internally by `lbexec` to parse the YAML
* A `DaVinci.make_config` function which is the nominal way people should be using DaVinci
* 931f514928ffbc16317a4c033b0a872b89f9855a makes it so that the global `DaVinci.options` object is never relied upon (as it will be removed once the transition to lbexec is complete)
* 7d2f007bba4eaf7467724e8c03da8acd5ec66b62 contains an example of porting to lbexec with the `test_davinci_tupling_All` test. If everyone is happy with it I'll port the remaining tests and start cleaning up.
To see how this might actually look in reality, make a yaml file named `options.yaml` containing (most of these keys probably aren't needed):
```yaml
input_files:
- root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp3/tests/spruce_all_lines_realtimereco_newPacking.dst'
annsvc_config: 'root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp3/tests/spruce_all_lines_realtime_newPacking.tck.json'
data_type: Upgrade
input_type: ROOT
simulation: true
conddb_tag: sim-20171127-vc-md100
dddb_tag: dddb-20171126
histo_file: 'sprucing_histos.root'
input_raw_format: 0.3
lumi: false
ntuple_file: 'sprucing_tuple.root'
print_freq: 1
process: 'Spruce'
stream: 'default'
```
You can then run davinci using: `lbexec DaVinciExamples.tupling.AllFunctors:alg_config options.yaml`
I think it would be good to sit down and discuss what should be stored in the `options.yaml` and what the arguments to `DaVinci.make_config` should be. In particular I think things like `process`/`stream`/`lumi`/`evt_pre_filters` might be worth moving to the Python side.
Work towards DPA grand task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/180 on configuration. Necessary for https://gitlab.cern.ch/lhcb-dpa/analysis-productions/LbAnalysisProductions/-/issues/68.https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/591Improving design of DV add_filter function2021-12-15T11:25:43+01:00Davide FazziniImproving design of DV add_filter functionRedesign of the add_filter function, removing the instantiation of all the HltAnn decoders.
Work on DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/180.Redesign of the add_filter function, removing the instantiation of all the HltAnn decoders.
Work on DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/180.Davide FazziniDavide Fazzinihttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/971Implement consistency checks in configuration of jobs on Spruced data2023-10-30T11:55:52+01:00Eduardo RodriguesImplement consistency checks in configuration of jobs on Spruced dataAs per our discussions @nskidmor and @pkoppenb.
This partially addresses issue https://gitlab.cern.ch/lhcb/DaVinci/-/issues/143, @cburr, as far as DaVinci jobs are concerned. Of course one should still address separately (guess in Gaudi...As per our discussions @nskidmor and @pkoppenb.
This partially addresses issue https://gitlab.cern.ch/lhcb/DaVinci/-/issues/143, @cburr, as far as DaVinci jobs are concerned. Of course one should still address separately (guess in Gaudi) the fact that EventSelector is not safe enough.Eduardo RodriguesPatrick KoppenburgEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/499Implementation of advanced features in Upgrade Configurable2021-02-25T09:59:18+01:00Davide FazziniImplementation of advanced features in Upgrade ConfigurableImplementing advanced features in DaVinci Configurable for Upgrade.
List of the new features:
- enabling the possibility to pass arguments from the command line
- adding a new DaVinci property for passing the python file containing the u...Implementing advanced features in DaVinci Configurable for Upgrade.
List of the new features:
- enabling the possibility to pass arguments from the command line
- adding a new DaVinci property for passing the python file containing the user algorithms
- adding the configuration of unpacking and MC association for simulated samples;
(for the moment it is implemented similarly to the old configuration, it will be updated in a future MR. See https://gitlab.cern.ch/lhcb-dpa/project/-/issues/63)
- adding functions for propagating the FSRs
- getting rid of the GaudiSequences
- implement the Moore selection framework
- adding a few extra useful functions
Further addressing https://gitlab.cern.ch/lhcb-dpa/project/-/issues/48.Eduardo RodriguesPatrick KoppenburgDavide FazziniEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/458Further move bits from AnalysisConf to the DaVinci configurable2020-11-10T11:36:56+01:00Eduardo RodriguesFurther move bits from AnalysisConf to the DaVinci configurableDepends on https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/713.Depends on https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/713.Patrick KoppenburgPatrick Koppenburghttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/692For truth matching make Spruce as the default root_in_tes2022-05-10T17:59:04+02:00Abhijit MathadFor truth matching make Spruce as the default root_in_tesDriven by this [discussion](https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/688#note_5548301), the MR has following changes:
- Change the default `root_in_tes` to be `Spruce`.
- The `configured_MCTruthAndBkgCatAlg` takes `process`...Driven by this [discussion](https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/688#note_5548301), the MR has following changes:
- Change the default `root_in_tes` to be `Spruce`.
- The `configured_MCTruthAndBkgCatAlg` takes `process` as input, instead of `root_in_tes`. The `root_in_tes` is set internally depending on `process`.Abhijit MathadAbhijit Mathadhttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1013Follow up on particle PV pointer, offline PV unbiasing example2024-03-22T16:00:46+01:00Wouter Hulsbergenwouterh@nikhef.nlFollow up on particle PV pointer, offline PV unbiasing exampleTo go with LHCb!4299, Rec!3665, Moore!2658
Example of PV unbiasing, using PV constraints in DTF.
Closing Moore#608To go with LHCb!4299, Rec!3665, Moore!2658
Example of PV unbiasing, using PV constraints in DTF.
Closing Moore#608https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1045Follow-up from https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/44042024-03-05T19:20:08+01:00Eduardo RodriguesFollow-up from https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4404FYI @graven and @eleckste.
While at it I believe I got rid of some trivial failures sitting there in other platforms since a while.FYI @graven and @eleckste.
While at it I believe I got rid of some trivial failures sitting there in other platforms since a while.Eduardo RodriguesEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/497Follow snake_case naming conventions rather than camelCase2021-01-28T19:16:09+01:00Eduardo RodriguesFollow snake_case naming conventions rather than camelCaseI noticed that we're not really following what seems to be the preferred naming conventions for upgrade code, hence this MR to improve on that from the onset. This is just a few first changes to make sure everyone is happy - @pkoppenb an...I noticed that we're not really following what seems to be the preferred naming conventions for upgrade code, hence this MR to improve on that from the onset. This is just a few first changes to make sure everyone is happy - @pkoppenb and @dfazzini.
I'm profiting from the occasion to make the main configuration functions available from the top level, to run as `gaudirun.py DaVinci:data` rather than `gaudirun.py DaVinci.ConfigurationUpgrade:data`, as agreed.
Hopefully we can converge here and have it all merged for Monday for the hackathon.Patrick KoppenburgDavide FazziniPatrick Koppenburghttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/735Follow lbexec changes2022-10-10T11:45:07+02:00Chris BurrFollow lbexec changesGoes alongside https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3725Goes alongside https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3725Chris BurrChris Burrhttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1061Follow https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4496 (Remove data_ty...2024-03-26T10:00:02+01:00Abhijit MathadFollow https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4496 (Remove data_type option)
Test with: https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4496, https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/3209, https://gitlab.cern.ch/lhcb-datapkg/AppConfig/-/merge_requests/233
Closes https://gitlab.cern.ch/lhcb/DaVinci/-...
Test with: https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4496, https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/3209, https://gitlab.cern.ch/lhcb-datapkg/AppConfig/-/merge_requests/233
Closes https://gitlab.cern.ch/lhcb/DaVinci/-/issues/176Abhijit MathadAbhijit Mathadhttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/995Follow https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4356 and https://git...2023-12-07T10:43:50+01:00Abhijit MathadFollow https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4356 and https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/1022To be tested with: https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4356 and https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/1022To be tested with: https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4356 and https://gitlab.cern.ch/lhcb/Analysis/-/merge_requests/1022Abhijit MathadAbhijit Mathadhttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/644Fix typo in jobOptions-Example.py2022-02-04T16:59:16+01:00Eduardo RodriguesFix typo in jobOptions-Example.pyDoes this mean that this file is not tested anywhere @dfazzini?Does this mean that this file is not tested anywhere @dfazzini?Eduardo RodriguesPatrick KoppenburgEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/506Fix Phys/DaVinci for Python 32021-04-21T16:17:53+02:00Eduardo RodriguesFix Phys/DaVinci for Python 3FYI @dfazzini.FYI @dfazzini.Eduardo RodriguesEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/602Fix issue for reading hlt2 files2021-12-14T09:20:39+01:00Davide FazziniFix issue for reading hlt2 filesFix issue related to the wrong unpacking of the hlt2 mc particles and vertices. To be merged together to LHCb!3352.
A new qmtest is added for reading a hlt2 .dst and creating a tuple with FunTuple.
Work on DPA task https://gitlab.cern.c...Fix issue related to the wrong unpacking of the hlt2 mc particles and vertices. To be merged together to LHCb!3352.
A new qmtest is added for reading a hlt2 .dst and creating a tuple with FunTuple.
Work on DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/180. Somewhat related to https://gitlab.cern.ch/lhcb-dpa/project/-/issues/179.Davide FazziniDavide Fazzinihttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1033Fix .gitlab-ci.yml for the build to work with LCG 1052024-02-13T16:57:28+01:00Eduardo RodriguesFix .gitlab-ci.yml for the build to work with LCG 105Indeed better to fix the failing build asap and this bit part of https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1029 would take longer to merge. This can go in asap to fix the builds.Indeed better to fix the failing build asap and this bit part of https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1029 would take longer to merge. This can go in asap to fix the builds.Eduardo RodriguesEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/505Fixes of a test and many Flake8 warnings2021-02-26T16:15:45+01:00Eduardo RodriguesFixes of a test and many Flake8 warningsFixes a test following big MR https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/499 and gets rid of a whole bunch of Flake8 warnings one would get if switching linting for all Python files, which can be tested with
`flake8 --exclude ...Fixes a test following big MR https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/499 and gets rid of a whole bunch of Flake8 warnings one would get if switching linting for all Python files, which can be tested with
`flake8 --exclude '*.opts.py' --select=F,E71,E9,W1,W6 `find Phys/DaVinci -name '*.py'``.
Better only switch on the linting for all Python files once we converge on what PyConf & co business to keep, I guess, as otherwise there would be quite a few more things to fix.
Addressing partially https://gitlab.cern.ch/lhcb-dpa/project/-/issues/89.
FYI @dfazzini, @mamartin and @pkoppenb.Eduardo RodriguesEduardo Rodrigues