MooreAnalysis merge requestshttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests2023-12-19T15:24:32+01:00https://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/135Remove contents of MooreAnalysis after move of HltEfficencyChecker to DaVinci2023-12-19T15:24:32+01:00Ross John HunterRemove contents of MooreAnalysis after move of HltEfficencyChecker to DaVinciGoes either with or after Moore!2812 and DaVinci!1005
Followed the same procedure as the `Phys` retirement in Phys!1037, leaving the `ReleaseNotes` in place, and a `README.md` with explanation of the removal and instructions on rebasing...Goes either with or after Moore!2812 and DaVinci!1005
Followed the same procedure as the `Phys` retirement in Phys!1037, leaving the `ReleaseNotes` in place, and a `README.md` with explanation of the removal and instructions on rebasing to DaVinci.
Completes and closes #45.Ross John HunterRoss John Hunterhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/134Explicitly specify old CALO raw banks or old muon decoding in HltEfficiencyCh...2024-01-26T11:50:39+01:00Ross John HunterExplicitly specify old CALO raw banks or old muon decoding in HltEfficiencyChecker yaml optionsFYI @jagoodin
Goes with Moore!2811
Options like:
```python
from RecoConf.calorimeter_reconstruction import make_digits
make_digits.global_bind(calo_raw_bank=False)
from RecoConf.hlt1_muonid import make_muon_hits
make_muon_hits.globa...FYI @jagoodin
Goes with Moore!2811
Options like:
```python
from RecoConf.calorimeter_reconstruction import make_digits
make_digits.global_bind(calo_raw_bank=False)
from RecoConf.hlt1_muonid import make_muon_hits
make_muon_hits.global_bind(geometry_version=2)
```
were hard-coded into the yaml-wizard `options_template.py.jinja` template, meaning they'll be added silently for any user of HltEfficiencyChecker's wizard. This MR makes them explicit yaml options that need to be asked for. Documentation of these new options is added in the Moore MR. The explicit options are added to those tests that require it.
Also do a little cleanup where I saw it could be done.
With the Moore MR closes #44. ~~Marked as Draft while the Moore MR is draft~~. Confirmed all MooreAnalysis tests pass locally.Sebastien PonceRoss John HunterSebastien Poncehttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/133Removal of legacy software2023-12-15T20:08:18+01:00Patrick KoppenburgRemoval of legacy software`DecayTreeTuple` and other legacy software it relies on are removed.
Part of https://gitlab.cern.ch/lhcb/Rec/-/issues/358`DecayTreeTuple` and other legacy software it relies on are removed.
Part of https://gitlab.cern.ch/lhcb/Rec/-/issues/358Brij Kishor JashalBrij Kishor Jashalhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/132Move HltEffChecker scripts onto RDataFrame2023-12-15T17:38:10+01:00Jamie GoodingMove HltEffChecker scripts onto RDataFrameThis MR replaces the existing ROOT TTree interface of the HltEfficiencyChecker scripts (e.g. hlt_calculate_rates.py) with RDataFrame for speedup. This MR also contains a reworking of the fitting to allow the user to specify if plots shou...This MR replaces the existing ROOT TTree interface of the HltEfficiencyChecker scripts (e.g. hlt_calculate_rates.py) with RDataFrame for speedup. This MR also contains a reworking of the fitting to allow the user to specify if plots should be produced per denominator or with all denominators shown together (thanks @nschulte for the idea and implementation!).
The performance numbers can be found posted below and performance checks are summarised in the checklist below:
- [x] Performance comparable with ``master``
- [x] Efficiencies consistent with ``master``
- [x] Overlaps consistent with ``master``
- [x] Rates consistent with ``master``Eduardo RodriguesBrij Kishor JashalJamie GoodingEduardo Rodrigueshttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/131Follow changes in Moore!21662024-01-26T11:50:39+01:00Gerhard RavenFollow changes in Moore!2166- replace make_RetinaCluster_raw_event with make_RetinaCluster_raw_bank- replace make_RetinaCluster_raw_event with make_RetinaCluster_raw_bankhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/130Follow Moore!28022024-01-09T15:50:18+01:00Sevda EsenFollow Moore!2802Adapt HltEfficiencyChecker config to use Streams introduced in Moore!2802Adapt HltEfficiencyChecker config to use Streams introduced in Moore!2802Rosen MatevRosen Matevhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/129Minor code improvements2023-10-31T09:10:40+01:00Marco Clemencicmarco.clemencic@cern.chMinor code improvements- fixed CMake configuration to be able to build in a *super project* context
- fix standalone compilation of header `isStable.h`- fixed CMake configuration to be able to build in a *super project* context
- fix standalone compilation of header `isStable.h`https://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/128Disable tests in dd4hep builds2023-08-24T15:00:07+02:00Christopher Rob Jonesjonesc@hep.phy.cam.ac.ukDisable tests in dd4hep buildsSee lhcb/MooreAnalysis#43See lhcb/MooreAnalysis#43https://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/127Follow change of default muon geometry version (Moore!2424)2023-07-14T20:57:44+02:00Rosen MatevFollow change of default muon geometry version (Moore!2424)In addition, fix a couple of occasions of a wrong calo_raw_bank setting
(leading to no calo objects being found)
@rjhunter sorry for missing this originally. Now the tests should work again.In addition, fix a couple of occasions of a wrong calo_raw_bank setting
(leading to no calo objects being found)
@rjhunter sorry for missing this originally. Now the tests should work again.Rosen MatevRosen Matevhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/126split track containers and persistency (versioning)2023-08-07T10:41:31+02:00Maarten Van Veghelsplit track containers and persistency (versioning)See for info https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2338
Goes with https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4127, https://gitlab.cern.ch/lhcb/Rec/-/merge_requests/3423, https://gitlab.cern.ch/lhcb/Moore/-/merge_requ...See for info https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2338
Goes with https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4127, https://gitlab.cern.ch/lhcb/Rec/-/merge_requests/3423, https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2338, https://gitlab.cern.ch/lhcb/Alignment/-/merge_requests/394, https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/926, https://gitlab.cern.ch/lhcb/MooreOnline/-/merge_requests/268RTA/2023.07.31https://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/125Fix of header compilation warnings2023-07-10T16:29:41+02:00Sebastien PonceFix of header compilation warningshttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/124HltEfficiencyChecker: HLT2 reconstruction configuration becomes configurable;...2023-08-03T14:54:33+02:00Ross John HunterHltEfficiencyChecker: HLT2 reconstruction configuration becomes configurable; set default to fastest; run_reco by default; more accurate HLT1-filtering output rateBuilt on top of !122. Goes with Moore!2419
This MR does a number of things:
* Makes running of the HLT2 reconstruction the default,
* Requires now that you specify a `hlt2_configuration` in the `yaml` wizard, which can either be `fastes...Built on top of !122. Goes with Moore!2419
This MR does a number of things:
* Makes running of the HLT2 reconstruction the default,
* Requires now that you specify a `hlt2_configuration` in the `yaml` wizard, which can either be `fastest` or `light_without_UT`. These correspond to `make_fastest_reconstruction` and `make_light_reco_pr_kf_without_UT` respectively.
* Make the former ("fastest") the new default in the options and documentation (closes #35).
* Replace `--using-hlt1-filtered-MC` with `--input-rate`. The former used to be a flag that would internally set the input rate of a min bias sample to 1 MHz (expected HLT1 rate). Recent discussions in #42 have pointed out that both the two usual min. bias samples are significantly far away from 1 MHz. So now provide the rate directly, and set it to 1.65 MHz in the case of `upgrade_minbias_hlt1_filtered`.
* Added new example options files `hlt2_rate_example_withGEC.yaml` and `hlt2_rate_hlt1latestBWdiv.py` to showcase how to get reliable HLT2 rates out of the `UpgradeHLT1FilteredWithGEC` min. bias, and also the latest HLT1 BW division samples.
Showcased in RTA/DPA General on 16/06/23. [Slides](https://indico.cern.ch/event/1289622/#3-hlt2-rate-testing).Yang GaoRoss John HunterYang Gaohttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/123Draft: Prevent output_file from being set when running on chained HLT2023-12-08T17:09:41+01:00Louis Henrylouis.henry@cern.chDraft: Prevent output_file from being set when running on chained HLTSee https://gitlab.cern.ch/lhcb/MooreAnalysis/-/issues/40#note_6728519See https://gitlab.cern.ch/lhcb/MooreAnalysis/-/issues/40#note_6728519Louis Henrylouis.henry@cern.chLouis Henrylouis.henry@cern.chhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/122Migrate HltEfficiencyChecker tupling to FunTuple2023-12-14T17:29:43+01:00Ross John HunterMigrate HltEfficiencyChecker tupling to FunTupleGoes with Rec!3415, Analysis!985 and Moore!2326.
`HltEfficiencyChecker` writes tuples in order to evaluate efficiencies and rates, which it currently does with legacy `TupleTool`s and `{MC}DecayTreeTuple`. This MR migrates that tupling ...Goes with Rec!3415, Analysis!985 and Moore!2326.
`HltEfficiencyChecker` writes tuples in order to evaluate efficiencies and rates, which it currently does with legacy `TupleTool`s and `{MC}DecayTreeTuple`. This MR migrates that tupling to use FunTuple.
Most of the functionality was already available with the existing FunTuple + functor framework. The two `MCTupleToolTrueSimEffHLT{1,2}` tuple tools that I wrote especially for HltEfficiencyChecker needed porting to algorithms, which is done in the Rec and Analysis MRs. I couldn't yet get the HLT1 version of this working (see #41), but I left the skeleton algorithm and functor there and will follow it up. I thought better to get the core HLT1 HltEfficiencyChecker functionality fixed sharpish, and fix the more niche case of HLT1 TrueSim effs later.
This MR importantly also fixes recent overestimation of HLT1 rates and efficiencies by restoring the control flow to `NONLAZY_AND`, at the expense of no Hlt1 True Sim efficiencies. This is being followed up in #41.
I've verified that HLT2 rates and efficiencies are unchanged under this migration, as they should be.
Will clear the way to the end of `{MC}DecayTreeTuple`.
TODO:
* [x] Docs update (Moore MR)
* [x] Verify results don't change on a few test cases.
* [x] ~~Finish off necessary refactor of analysis scripts to adapt to branch naming changes~~,
* [x] Check can run happily with NONLAZY
* [x] (here or follow-up) true-sim matching tools -> algorithms? (Rec & Analysis MRs)
* [x] Check on status of #36 when ready to merge. Doesn't fix that one unfortunately.
Follow-ups:
* Smarter tests to ensure recent breaks dont occur again,
* Refactor the analysis scripts to break up into smaller functions,
* Retire MCDecayTreeTuple and TupleTools in MooreAnalysis...
* ...after above; move HltEfficiencyChecker to DaVinci and obviate MooreAnalysis?
Closes #19 #7Rosen MatevMilosz ZdybalRoss John HunterRosen Matevhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/121Remove all tests as they depend on FilterDesktop2023-05-15T20:04:24+02:00Patrick KoppenburgRemove all tests as they depend on FilterDesktopFilterDesktop is being removed as part of Rec#358 . =HltIntegrationTests= depends on it and hence it is removed. Please fix and re-add in a separate MR if it still serves a purpose. The project https://gitlab.cern.ch/lhcb/LHCbIntegration...FilterDesktop is being removed as part of Rec#358 . =HltIntegrationTests= depends on it and hence it is removed. Please fix and re-add in a separate MR if it still serves a purpose. The project https://gitlab.cern.ch/lhcb/LHCbIntegrationTests (see [Davide's talk](https://indico.cern.ch/event/1272210/#5-lhcb-integration-tests)) could be a better place.
Eventually Rec#358 will lead to the removal of DaVinciAlgorithm and thus DecayTreeTuple. This will make MooreAnalysis an empty shell. I suggest to terminate this project asap (not first time I say that 😉).
Needs Rec!3392 Moore!2259 Moore!2248 !119Izaac SanderswoodIzaac Sanderswoodhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/120Refactor HltEfficiencyChecker config to reduce duplication and prepare for Fu...2023-05-16T16:52:24+02:00Ross John HunterRefactor HltEfficiencyChecker config to reduce duplication and prepare for FunTuple migrationGoes with Moore!2274 for documentation updates.
* Isolate tupling to one common function for easy replacement with FunTuple,
* Reduce code duplication and break up the big functions a little,
* Apply suggested cleanups to close #27.
T...Goes with Moore!2274 for documentation updates.
* Isolate tupling to one common function for easy replacement with FunTuple,
* Reduce code duplication and break up the big functions a little,
* Apply suggested cleanups to close #27.
TODO:
* [x] Docs update MR in Moore,
* [x] Fill in doc strings,
* [x] Unify Allen configuration between two `run_` functions? Not clear why they have to be different.
Closes #27.Izaac SanderswoodRoss John HunterIzaac Sanderswoodhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/119Move to more modern Starterkit Bs -> Jpsi Phi line in HltEfficiencyChecker ex...2023-05-15T20:04:24+02:00Ross John HunterMove to more modern Starterkit Bs -> Jpsi Phi line in HltEfficiencyChecker examplesGoes with Moore!2259.
The legacy `Hlt2BsToJpsiPhi` is based on LoKi and therefore being removed, so this MR mainly updates to use the more modern Starterkit version of this line.
Addresses one half of Moore#469 (Moore!2248 will do the...Goes with Moore!2259.
The legacy `Hlt2BsToJpsiPhi` is based on LoKi and therefore being removed, so this MR mainly updates to use the more modern Starterkit version of this line.
Addresses one half of Moore#469 (Moore!2248 will do the rest), and closes #38.Ross John HunterRoss John Hunterhttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/118Adaptations for new MC samples in tests with data-like decoding versions2023-08-03T14:54:33+02:00Arthur Marius HennequinAdaptations for new MC samples in tests with data-like decoding versionsGoes with https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/1703Goes with https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/1703https://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/117Resolve "Issues with plotting overlap metrics and saving intersections in Hlt...2023-12-11T11:00:03+01:00Jamie GoodingResolve "Issues with plotting overlap metrics and saving intersections in HltEfficiencyChecker"Resolves the bug raised by @miolocco in plotting labels of overlap metrics by HltEfficiencyChecker, #39. Also resolves the bug in writing of intersections (also described in #39).Resolves the bug raised by @miolocco in plotting labels of overlap metrics by HltEfficiencyChecker, #39. Also resolves the bug in writing of intersections (also described in #39).Sebastien PonceJamie GoodingSebastien Poncehttps://gitlab.cern.ch/lhcb/MooreAnalysis/-/merge_requests/116Don't run tuples if the event was not selected by hlt12024-03-13T17:50:49+01:00Arthur Marius HennequinDon't run tuples if the event was not selected by hlt1Goes with https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2200Goes with https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2200