Allen merge requestshttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests2022-06-22T18:50:27+02:00https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/913Refactor Allen geometries2022-06-22T18:50:27+02:00Daniel Campora PerezRefactor Allen geometries* A single Allen geometry is now included with Allen,
located under folder `input/detector_configuration`.
* Geometries now exist under folder `/scratch/allen_geometries`.
* Names of folders in `/scratch/allen_geometries` match the key...* A single Allen geometry is now included with Allen,
located under folder `input/detector_configuration`.
* Geometries now exist under folder `/scratch/allen_geometries`.
* Names of folders in `/scratch/allen_geometries` match the key
from TestDB.
* Geometries now also exist in eos, under
`/eos/lhcb/wg/rta/WP6/Allen/geometries`.
* Documentation has been updated accordingly.
* CI jobs have been updated to pick up the geometries from
the updated folder structure.https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/948Redefine the global event cut to not use UT information2023-03-22T18:45:39+01:00Vladimir GligorovRedefine the global event cut to not use UT informationThis removes the use of UT information from the global event cut. The default cut value is not changed which as a byproduct means that the GEC in Allen will become more of a "sanity" cut removing only the really few very highest occupanc...This removes the use of UT information from the global event cut. The default cut value is not changed which as a byproduct means that the GEC in Allen will become more of a "sanity" cut removing only the really few very highest occupancy events, which probably goes in the direction we want to go long-term anyway ( @mvesteri @decianm ). This will be a mess of reference updates and I may well have forgotten something but otherwise is ready for review @dovombru @dcampora @raaij
With this MR, the following slowdown is observed due to loosening the GEC:
```
Device Throughput (kHz) Reference Throughput (kHz) Speedup % change Status
-------------------------- ------------------ ---------------------------- --------- ---------- ---------
NVIDIA RTX A5000 136.13 149.7 0.91x -9.07% DECREASED
AMD EPYC 7502 32-Core 16.72 17.27 0.97x -3.20% OK
NVIDIA GeForce RTX 2080 Ti 110.87 120.8 0.92x -8.22% DECREASED
NVIDIA GeForce RTX 3090 176.63 194.07 0.91x -8.98% DECREASED
Device-averaged speedup: 0.93x
% change: -7.37%
status: DECREASED
```Roel AaijRoel Aaijhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1019Muon standalone reconstruction2022-11-22T16:44:27+01:00Adrian Casais VidalMuon standalone reconstructionExtraction from Allen!847 of the muon stubs reconstruction, which can be tested on data in the pit
The throughput of find_muon_hits according [to the profiling](https://mattermost.web.cern.ch/lhcb/pl/cnkurp4dqpri3crs6buqqah4dh) is
```
...Extraction from Allen!847 of the muon stubs reconstruction, which can be tested on data in the pit
The throughput of find_muon_hits according [to the profiling](https://mattermost.web.cern.ch/lhcb/pl/cnkurp4dqpri3crs6buqqah4dh) is
```
Throughput of branch samarian_acasaisv_add_Mustandalone_rec (a8c23490), sequence hlt1_pp_default over dataset upgrade_mc_minbias_scifi_v5_retinacluster_000_v1 build options default:
find_muon_hits │▌ 0.32 %
consolidate_muon │▌ 0.01 %
```Rosen MatevRosen Matevhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1123Data quality validator2024-02-22T10:37:53+01:00Andrew George MorrisData quality validatorThis adds the various (modular) DataQualityValidator algorithms into `device/validators` along with their relevant functions in `HLT1.py`.
They are designed in a modular way such that more information can be simply added later if necess...This adds the various (modular) DataQualityValidator algorithms into `device/validators` along with their relevant functions in `HLT1.py`.
They are designed in a modular way such that more information can be simply added later if necessary.
Output is done to the monitoring file via the ROOT service.
This MR has become dependent on !1397 and MooreOnline!314Christina AgapopoulouAndrew George MorrisChristina Agapopoulouhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1150Move CI to RHEL9 and CUDA 12.12023-05-01T18:19:42+02:00Ryunosuke O'NeilMove CI to RHEL9 and CUDA 12.1This MR moves the Allen CI to RHEL 9 and CUDA 12.1.
- Uses the CERN provided docker image for alma linux 9 `gitlab-registry.cern.ch/linuxsupport/alma9-base` in build jobs
- Uses the target `x86_64_v3-el9-gcc12+cuda12_1` from `LCG103`.
-...This MR moves the Allen CI to RHEL 9 and CUDA 12.1.
- Uses the CERN provided docker image for alma linux 9 `gitlab-registry.cern.ch/linuxsupport/alma9-base` in build jobs
- Uses the target `x86_64_v3-el9-gcc12+cuda12_1` from `LCG103`.
- A small reference update due to the change of compiler to CUDA 12.1 was necessary.
Will come at a later stage:
- There are no `clang12` platforms in LCG 103 yet, so for the moment we will stick to gcc12 and gcc12+cuda12_1 builds.
- A new HIP installation is in the works, we will use HIP 5.4.3 once it is available in CVMFS.
The impact in speed is the following:
```
Device Throughput (kHz) Reference Throughput (kHz) Speedup % change Status
-------------------------- ------------------ ---------------------------- --------- ---------- ---------
NVIDIA RTX A5000 120.37 123.18 0.98x -2.28% OK
AMD EPYC 7502 32-Core 15.31 16.56 0.92x -7.54% DECREASED
NVIDIA GeForce RTX 2080 Ti 97.73 99.85 0.98x -2.12% OK
NVIDIA GeForce RTX 3090 156.05 154.88 1.01x 0.76% OK
```
It is clear that the impact in CPU performance comes from moving from clang to gcc. In the GPU case this would have to be studied better.
The simplified Allen CI passes. The full Allen CI passes with one test exception, see issue https://gitlab.cern.ch/lhcb/Allen/-/issues/413https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1179Fix tae_index increment bug2023-04-17T08:10:34+02:00Kate Abigail RichardsonFix tae_index increment bugWe were able to reproduce the new TAE sequence crash offline with a recent MEP which led us to find this bug in the OutputHandler. Testing offline there is no longer a crash.We were able to reproduce the new TAE sequence crash offline with a recent MEP which led us to find this bug in the OutputHandler. Testing offline there is no longer a crash.https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1186Add monitoring using global atomicity histograms2023-08-03T14:54:33+02:00Kate Abigail RichardsonAdd monitoring using global atomicity histogramsAdds monitoring of HLT1 using Gaudi accumulator histograms which are `atomicity::none` in a wrapper to lock them at the histogram level instead of the bin level as `atomicity::full` does. Currently this is done in Allen so that it can be...Adds monitoring of HLT1 using Gaudi accumulator histograms which are `atomicity::none` in a wrapper to lock them at the histogram level instead of the bin level as `atomicity::full` does. Currently this is done in Allen so that it can be added quickly, but we will move it to Gaudi in case any other applications want to use this scheme. Using the single node server to test throughput, this performs within 7.5% of master as allowed by the CI tests, so I do not expect a slowdown. This includes all histograms from the `monitoring_pit` branch used at the end of 2022 and the ones included in @dcraik's original merge request except for the 2D CALO ones. The histograms added are:
Reconstruction:
- MakeLongTrackParticles (number of tracks, track eta, track phi, track pt)
- pv_beamline_cleanup (number of PVs, PV x, PV y, PV z)
- ConsolidateSciFi (number of tracks, track eta, track phi, number of hits)
- ConsolidateSciFiSeeding (number of seeds, track eta, track phi, number of hits)
- ConsolidateMatchedTracks (number of tracks, track eta, track phi, number of hits)
- VeloConsolidateTracks (number of tracks)
- VeloKalmanFilter (track eta, track phi, number of hits)
- VertexFitter (number of SVs)
Lines:
- D2KPi (mass, pt)
- D2KK (mass, pt)
- D2PiPi (mass, pt)
- LowMassNoipDielectronLine (mass with and without brem correction, same sign and opposite sign)
- KsToPiPi (mass, pt)
- DiMuonDrellYanLine (same sign and opposite sign mass)
- DiMuonMassLine (J/Psi mass)
- DiMuonNoIPLine (same sign and opposite sign mass)
The prescales for LowMassNoipDielectronLine and DiMuonNoIPLine were switched to postscales to allow for those events to fill the monitoring histograms with no change to the overall rate. Once the method is finalized, I will create documentation on how to add monitoring. Many thanks to @dcampora, @nnolte, @raaij, and everyone else who advised on this.RTA/2023.06.1https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1193Prepare for upcoming changes in GaudiAlg2023-05-23T16:28:25+02:00Marco Clemencicmarco.clemencic@cern.chPrepare for upcoming changes in GaudiAlgThis is a backward compatible change needed to prepare for gaudi/Gaudi!1444.
- explicitly include GaudiAlg/FunctionalUtilities.h and
Gaudi/Accumulators.h
- explicitly set Gaudi::Functional base class with useLegacyAlgorithm or
useAl...This is a backward compatible change needed to prepare for gaudi/Gaudi!1444.
- explicitly include GaudiAlg/FunctionalUtilities.h and
Gaudi/Accumulators.h
- explicitly set Gaudi::Functional base class with useLegacyAlgorithm or
useAlgorithm where needed
Part of a set of independent MRs: Rec!3395 Allen!1193 Analysis!975 Boole!466 LHCb!4082https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1206Update References for: Allen!1129, Moore!2129, MooreAnalysis!114 based on lhc...2023-05-22T12:10:23+02:00Software for LHCbUpdate References for: Allen!1129, Moore!2129, MooreAnalysis!114 based on lhcb-master-mr/7933https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1216Added missing default initial values for PV classes2023-05-22T16:43:38+02:00Patrick SpradlinAdded missing default initial values for PV classes## Related
Follow-up to merge of https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/794 and https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2278.
Related to https://gitlab.cern.ch/lhcb/Allen/-/issues/422, **but does not close it.**
...## Related
Follow-up to merge of https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/794 and https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2278.
Related to https://gitlab.cern.ch/lhcb/Allen/-/issues/422, **but does not close it.**
## Description
Following the merge of https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/794 and https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2278, the RTA Software Manager noted that the new test introduced by https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2278, `RecoConf.allen_gaudi_pv_with_mcchecking`, produced unstable stdout for some architectures (see [the discussion on the MR](https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/794#note_6729964)).
This instability was traced to unitialized date members in Allen's PV class. This MR provides default initialization for those data members, which should stabilize the test.
In local testing, this MR successfully stabilized the output of `RecoConf.allen_gaudi_pv_with_mcchecking` on the `x86_64_v2-centos7-clang12+detdesc-opt` platform without the need for an updated reference. A `ci-test` is required to check whether other platforms may require a reference update.Roel AaijRoel Aaijhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1222Fix UndefinedBehaviourSanitizer runtime errors2023-07-17T09:00:20+02:00Da Yu TouFix UndefinedBehaviourSanitizer runtime errorsFixes the runtime errors in https://gitlab.cern.ch/lhcb/Allen/-/issues/431. Solves problem 2 in https://gitlab.cern.ch/lhcb/Allen/-/issues/263.
1. Memory alignment errors in are fixed with `memcpy`.
2. Workaround negative value bit shi...Fixes the runtime errors in https://gitlab.cern.ch/lhcb/Allen/-/issues/431. Solves problem 2 in https://gitlab.cern.ch/lhcb/Allen/-/issues/263.
1. Memory alignment errors in are fixed with `memcpy`.
2. Workaround negative value bit shift left error by reinterpreting initial `int16` as `uint16` and applying left bit shift on resulting positive integer. (I can provide a test script to confirm the behaviour is the same as before)RTA/2023.07.31Rosen MatevDaniel Campora PerezRosen Matevhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1224Run dump_geometry test on simulation to be consistent between DetDesc and DD4hep2023-05-30T16:53:30+02:00Roel AaijRun dump_geometry test on simulation to be consistent between DetDesc and DD4hepThe same reference is intended to be used between DetDesc and DD4hep to test some measure of consistency between the builds. Such a test should use simulation to avoid failing on divergences between SIMCOND and the run 3 conditions DB.The same reference is intended to be used between DetDesc and DD4hep to test some measure of consistency between the builds. Such a test should use simulation to avoid failing on divergences between SIMCOND and the run 3 conditions DB.Sebastien PonceSebastien Poncehttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1243decoding MuonError banks2024-02-22T12:26:00+01:00Marco Santimariadecoding MuonError banksThis change enables the "MuonError" banks to be decoded along with the regular "Muon" banks, following [this](https://gitlab.cern.ch/lhcb/LHCb/-/blob/add_stuff_muon_decoding/Muon/MuonDAQ/src/components/MuonRawInUpgradeToHits.cpp?ref_type...This change enables the "MuonError" banks to be decoded along with the regular "Muon" banks, following [this](https://gitlab.cern.ch/lhcb/LHCb/-/blob/add_stuff_muon_decoding/Muon/MuonDAQ/src/components/MuonRawInUpgradeToHits.cpp?ref_type=heads) to recover data from desynched Tell40RTA/2023.06.0Roel AaijRosen MatevRoel Aaijhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1252A/C side velo tracks split for the PV reconstruction2024-02-19T12:30:15+01:00Bogdan KutsenkoA/C side velo tracks split for the PV reconstruction- New make_velo_tracks_ACsplit allows to split tracks on A/C Velo sides based on the velo module of the hits
- The new CUDA algorithms tracks_ACsplit_counters_t and tracks_ACsplit_t - are placed in between the reconstruction of velo tr...- New make_velo_tracks_ACsplit allows to split tracks on A/C Velo sides based on the velo module of the hits
- The new CUDA algorithms tracks_ACsplit_counters_t and tracks_ACsplit_t - are placed in between the reconstruction of velo tracks and consolidation
- Both velo_view objects from A and C sides successfully pass the velo tracks validation check
- Addition of A/C velo split to the main sequence resulted in ~6% throughput decrease https://gitlab.cern.ch/lhcb/Allen/-/jobs/30696653. Therefore separate sequence velo_ACsplit_validation.py was created and efficiency test was added to the full pipeline
Questions:
1. Philosophical question. What is the most optimized strategy for the determination of the A/C side? Right now the side is determined by the number of hits corresponding to it. Belonging of the hit to the side is obtained from a Velo module. Tracks with equal number of hits are removed
2. What is the reason for the dev_tracks array (Velo::TrackHits) and dev_three_hit_tracks_output (Velo::TrackletHits) from make_velo_tracks function to have a size, which is determined by dev_offsets_estimated_input_size? Why there is no additional prefix sum to determine the number of tracks and 3 hit tracks as a first step of make_velo_tracks , before velo_search_by_triplet_t https://gitlab.cern.ch/lhcb/Allen/-/blob/master/configuration/python/AllenConf/velo_reconstruction.py#178
3. Why in the main Allen sequence - hlt1_pp_default in the reconstruction step - hlt1_reconstruction, after making velo tracks velo_states are created and then make_pvs (in which velo_states are recreated again) https://gitlab.cern.ch/lhcb/Allen/-/blob/master/configuration/python/AllenConf/hlt1_reconstruction.py#L34. Probably more optimized way to do that is to provide velos_states as an input to make_pvs
Performance was presented in https://indico.cern.ch/event/1249024/#5-pv-reconstruction-in-hlt1-wi
Closes #383Christina AgapopoulouBogdan KutsenkoChristina Agapopoulouhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1381Add 2D zero suppression in decoding2023-12-22T19:05:20+01:00Jean-Francois MarchandAdd 2D zero suppression in decodingAdd 2D zero suppression in Allen decoding, should fix the differences observed between HLT2 and HLT1 (https://gitlab.cern.ch/lhcb/Allen/-/issues/464)
Default ADC cut set to 10 for the seed, and -5 for the neighbours, same as for HLT2Add 2D zero suppression in Allen decoding, should fix the differences observed between HLT2 and HLT1 (https://gitlab.cern.ch/lhcb/Allen/-/issues/464)
Default ADC cut set to 10 for the seed, and -5 for the neighbours, same as for HLT2Christina AgapopoulouArthur Marius HennequinChristina Agapopoulouhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1397Fix handling of shared_ptr to provider in allen.py2024-02-09T16:20:43+01:00Roel AaijFix handling of shared_ptr to provider in allen.pyThe `MDFProvider` was being destructed after the device had been reset.
A `shared_ptr<IInputProvider>` is used in `RuntimeOptions` to sort out memory management of the `TESProvider` in Allen-in-Moore. This was propagated to the `allen()...The `MDFProvider` was being destructed after the device had been reset.
A `shared_ptr<IInputProvider>` is used in `RuntimeOptions` to sort out memory management of the `TESProvider` in Allen-in-Moore. This was propagated to the `allen()` function, but there using a `shared_ptr` is the wrong type. This MR fixes the `allen()` signature to have a bare pointer instead and then sorts out the mismatch internally.
Together with: https://gitlab.cern.ch/lhcb/MooreOnline/-/merge_requests/314Sebastien PonceSebastien Poncehttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1398ElectronID NN and MVA machinery refactor2024-03-18T07:42:08+01:00Adrian Casais VidalElectronID NN and MVA machinery refactorThe main purpose of this MR is to add an NN operator for ElectronID necessary to correctly handle the background of `A' -> ee` search.
Refactors current implementation of `TwoTrack`,this refactoring is also applied to this operator.
Feat...The main purpose of this MR is to add an NN operator for ElectronID necessary to correctly handle the background of `A' -> ee` search.
Refactors current implementation of `TwoTrack`,this refactoring is also applied to this operator.
Features:
- Scifi extrpolation - ECAL barycenter
- Dispersion of the barycenter
- Boolean: is the main cell of the cluster a local maximum?
- EoP with selective matching
- EoP with 3x3 cluster
![delta_barycenter](/uploads/4b493dbc298be8b15e7bcca095ae4db8/delta_barycenter.png)
![dispersion_x](/uploads/0f0bb7d12f29c108015cf85905f11623/dispersion_x.png)
![dispersion_xy](/uploads/44ec4e9b69f4af815ab33a9372a7a2d3/dispersion_xy.png)
![dispersion_y](/uploads/db403e2b21c05710eb52127bc7a88ffb/dispersion_y.png)
![track_local_max](/uploads/86a54d4346e818267a43c2aacce14c8a/track_local_max.png)
![EoP](/uploads/92b3f890251a061c9eac21c785962723/EoP.png)
![EoP3x3](/uploads/8ef6fd90c39d8353fb9a3230007c011a/EoP3x3.png)
![corr_matrix](/uploads/d3a7b9f5d5a4b9982e2aa45315fda431/corr_matrix.png)
NN trained with 2024 expected MC:
- Upsilon -> ee
- Jpsi -> ee
- Ks-> ee
Crosschecked with DP (100 MeV) -> ee simulation
Expected reduction of `~50%` per electron. On 30/06 calib MEPs it shows to reduce the rate up to `80%`.
I also tuned the `LowMassDielectron` lines to fit better in the BW by using this operator.
The main todo point once this is approved is to migrate the `json` file to PARAMFILES and eliminate the CMakelists entry. Shipped with the MR for testing.
![rocpt_in_0.5_1__p_in_3.0_6.0](/uploads/211627c9d5e8b73da1a2e0c892b4875e/rocpt_in_0.5_1__p_in_3.0_6.0.png)![rocpt_in_0.5_1__p_in_6.0_10.0](/uploads/5b355135471677367480f44aefdc74a6/rocpt_in_0.5_1__p_in_6.0_10.0.png)![rocpt_in_1_1.6__p_in_3.0_6.0](/uploads/78d24ab940eb7197c2fd2ca31a163d78/rocpt_in_1_1.6__p_in_3.0_6.0.png)![rocpt_in_1_1.6__p_in_6.0_10.0](/uploads/43dcd83f79d1dcc647530cb636e7b973/rocpt_in_1_1.6__p_in_6.0_10.0.png)![rocpt_in_1.6_2.2__p_in_10.0_20.0](/uploads/278b3deb93536813418fa00b55159491/rocpt_in_1.6_2.2__p_in_10.0_20.0.png)![rocpt_in_1.6_2.2__p_in_20.0_40.0](/uploads/4c200d56fa868be4d4063da53ef30f13/rocpt_in_1.6_2.2__p_in_20.0_40.0.png)
Goes with https://gitlab.cern.ch/lhcb-datapkg/ParamFiles/-/merge_requests/72HLT1 reconstruction for first data-takingAndre GuntherAdrian Casais VidalYingao TangAndre Guntherhttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1415Configurable velo-UT Ghost Track Cut2024-02-19T15:34:52+01:00Da Yu TouConfigurable velo-UT Ghost Track CutThe cut on velo-UT ghost track is now a parameter that can be configured in python. This cut is a linear discriminat in `CompassUT` that uses reconstructed `p`, $p_{t}$ and pseudo track $\chi^{2}$.The cut on velo-UT ghost track is now a parameter that can be configured in python. This cut is a linear discriminat in `CompassUT` that uses reconstructed `p`, $p_{t}$ and pseudo track $\chi^{2}$.Sebastien PonceSebastien Poncehttps://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1428Add beam-beam filter to all lines that didn't have a filter on BX type yet.2024-03-18T10:46:54+01:00Roel AaijAdd beam-beam filter to all lines that didn't have a filter on BX type yet.Add a beam-beam ODIN filter to all lines that don't have specific other requirements.
Also add Hlt1ODINCalib that triggers on ODIN calibration triggers, plus routing bit 10 for Hlt1ODINCalib.
Closes #338Add a beam-beam ODIN filter to all lines that don't have specific other requirements.
Also add Hlt1ODINCalib that triggers on ODIN calibration triggers, plus routing bit 10 for Hlt1ODINCalib.
Closes #338RTA/2024.03.04https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1429Update routing bit 25 for events with error banks2024-02-13T14:43:09+01:00Roel AaijUpdate routing bit 25 for events with error banksChristina AgapopoulouChristina Agapopoulou