LHCb merge requestshttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests2024-03-28T11:51:08+01:00https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4404Add option to ANNSvc to publish decoding keys to monitoring hub, and read dec...2024-03-28T11:51:08+01:00Gerhard RavenAdd option to ANNSvc to publish decoding keys to monitoring hub, and read decoding keys from FSRAdd a property 'RegisterWithMonitoringHub' to ANNSvc which, if set to true, will register it as an entity with the monitoring hub. This allows any sink to pick up decoding keys seen by the ANNSvc and obtain a JSON representation of them....Add a property 'RegisterWithMonitoringHub' to ANNSvc which, if set to true, will register it as an entity with the monitoring hub. This allows any sink to pick up decoding keys seen by the ANNSvc and obtain a JSON representation of them.
Basically, implement this suggestion: https://gitlab.cern.ch/lhcb/LHCb/-/issues/300#note_6699226.
Needs https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1045.Sebastien PonceEduardo RodriguesNicole SkidmoreYingao TangSebastien Poncehttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4453Fix truth matching persistency bug2024-03-26T15:13:51+01:00Jiahui ZhuoFix truth matching persistency bugThe issue originated from https://gitlab.cern.ch/lhcb/Moore/-/issues/580, where @yihou identified anomalies in neutral truth matching. Subsequently, @alobosal confirmed the bug during the HLT2 line development. I suggested using the `MCR...The issue originated from https://gitlab.cern.ch/lhcb/Moore/-/issues/580, where @yihou identified anomalies in neutral truth matching. Subsequently, @alobosal confirmed the bug during the HLT2 line development. I suggested using the `MCReconstructed` tool to double-check the truth-matching relation table at the DaVinci level. Then, @yihou observed that all candidates had only one truth-matching result, which is uncommon for neutral particles.
The debugging process then began, and thanks a lot to @yihou for all the assistance with debugging! Eventually, the bug was identified. It was due to the recently implemented selective parker, which includes a mechanism to prevent duplicate entries based on the `from` candidate in the table. This results in a situation where if one `from` candidate is related to multiple `to` candidates, only the first entry persists, and the rest are removed by this kill-duplication mechanism. Consequently, the truth-matching table retains only one entry for each candidate (the first one, which, since the table is sorted by weight, corresponds to the worst candidate). This has a minor impact on charged particles, which usually have only one associated candidate, but it is disastrous for neutral particles, which typically have more than one associated candidate and the current implementation will only persist the worst one.
This MR addresses this issue and, so makes `MCReconstructed` useful.
FYI: @adeoyang @amathad @erodrigu @yihou @alobosal @gravenJiahui ZhuoJoshua Ethan HorswillJiahui Zhuohttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3528Enforce use of an encoding/decoding key (TCK) in all Hlt encoders/decoders, d...2024-03-26T09:55:16+01:00Gerhard RavenEnforce use of an encoding/decoding key (TCK) in all Hlt encoders/decoders, do not use online-reserved bits in SourceID, remove explicit mention of 'packed' locations, enable 'stable' persist reco locationsCloses #194, #220, #221, #225, #70 also see #202
- replace HltRawBankDecoderBase base class with a thread-safe service and
standalone utility funtions, and update decoder classes accordingly
- replace use of non-thread-safe Error and ...Closes #194, #220, #221, #225, #70 also see #202
- replace HltRawBankDecoderBase base class with a thread-safe service and
standalone utility funtions, and update decoder classes accordingly
- replace use of non-thread-safe Error and Warning with MsgCounter
- remove support of partially decoding the run1-style combined Hlt1+Hlt2 rawbanks
- put code in LHCb::Hlt::DAQ namespace
- use IIndexedANNSvc for both encoding and decoding to get various int <-> string mappings
** note: this includes the ExecutionReportsWriter in HLTScheduler!!!
- add GitANNSvc as new implementations of IIndexedANNSvc, and always push the relevant encoding table to the underlying git repository if not already present, using the git SHA1SUM to derive the encoding key, which is used as the label to look up the encoding table.
- GitANNSvc uses a 'stack' of git repositories, and will fall-back to the 'next' repo in case the requested oid cannot be found
- add the above encoding / decoding 'key' to PackedBuffer, and make sure it propagates when merging/splitting buffers. Add property to overrule the key obtained from the data (and add warnings if non-zero key in data is overruled, and if zero key is not overruled)
- do _not_ use the ANNSvc configurable as global variable written and read from a json file to record which TES locations + types are available on the output, and thus could (should) be configured in the read back application. Instead use a dedicated configuration-time generated json 'manifest' instead.
- no longer split the 'large' amalgamated buffer which lives in the DstData RawBank into smaller individual buffers for each container which each individually go in the TES at the 'packed' locations. Instead, transform the DstData raw bank(s) into a map of { locationID -> individual buffer } for all buffers present inside the amalgamated buffer, and have all unpackers use this single object as input, directly producing their output from it -- as a result, there are no 'packed' locations anymore... so also remove mention of those from the encoding table.
- enable the possibility to the explicitly specify 'stable' persist reco locations
Changes to the raw data bank format, and increment in the versions of Hlt{Sel,Dec}Reports and DstData
- HltSelReports, HltDecReports, DstData: re-define the sourceID in the header to avoid using the 5 most significant now-reserved bits
- HltDecReports: add the encoding key explicitly (in addition to the TCK), and verify that it reproduces the required int->string mapping
- DstData: add the compression and encoding keys to the payload, make the embedded locationIDs refer to the original (i.e. unpacked) TES locations
- adapt the decoding to deal with both old/new version
See also slides at https://indico.cern.ch/event/1179084/
Must be applied in conjunction with Allen!911 lhcb/DaVinci!733 lhcb/MooreAnalysis!91 Moore!1529 lhcb/MooreOnline!142Christopher Rob Jonesjonesc@hep.phy.cam.ac.ukGerhard RavenChristopher Rob Jonesjonesc@hep.phy.cam.ac.ukhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4338Persistency for ChargedPID, BremInfo and GlobalChargedPID classes2024-03-25T11:47:43+01:00Tara Nanut PetricPersistency for ChargedPID, BremInfo and GlobalChargedPID classesAdds persistency of PID objects from calorimeter and global PID variables (CombDLL/ProbNN) (similar to RichPID, MuonPID, ...) to improve consistency and allow packing choices per variable/object (not being forced to use float/double on d...Adds persistency of PID objects from calorimeter and global PID variables (CombDLL/ProbNN) (similar to RichPID, MuonPID, ...) to improve consistency and allow packing choices per variable/object (not being forced to use float/double on disk/memory resp.; also no need to save keys (32 bit)).
Removes additionalInfo (which is filled from these objects anyway) for this info and improves consistency between NeutralPID objects and the ones here. With these changes, no more additionalInfo in the ProtoParticles is filled.
Needs https://gitlab.cern.ch/lhcb/Rec/-/merge_requests/3740 https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/3149 https://gitlab.cern.ch/lhcb/DaVinci/-/merge_requests/1050 https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/merge_requests/63RTA/2024.03.27Andre GuntherAnfeng LiAndre Guntherhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4467Fixed packing of particle covariance matrix2024-03-25T10:58:18+01:00Wouter Hulsbergenwouterh@nikhef.nlFixed packing of particle covariance matrixFixes https://gitlab.cern.ch/lhcb/LHCb/-/issues/347
This also fixes a small issue with the computation of the energy in the unpacker. (This change is backward compatible.)
**Important note:** this updates the Particle packing, but it d...Fixes https://gitlab.cern.ch/lhcb/LHCb/-/issues/347
This also fixes a small issue with the computation of the energy in the unpacker. (This change is backward compatible.)
**Important note:** this updates the Particle packing, but it does not change the packing version as the target branch has not yet been used to reconstruct/generate data.RTA/2024.03.27Wouter Hulsbergenwouterh@nikhef.nlSevda EsenAndre GuntherAnfeng LiWouter Hulsbergenwouterh@nikhef.nlhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4444Support zstd compressed MDF reading in old-style IO2024-03-25T09:53:53+01:00Chris BurrSupport zstd compressed MDF reading in old-style IO~~A crude proof of concept for guiding the discussion in https://gitlab.cern.ch/lhcb/opg/-/issues/41. It seems to work for all of the offline use cases that we actually have at the moment.~~
See https://gitlab.cern.ch/lhcb/opg/-/issues/...~~A crude proof of concept for guiding the discussion in https://gitlab.cern.ch/lhcb/opg/-/issues/41. It seems to work for all of the offline use cases that we actually have at the moment.~~
See https://gitlab.cern.ch/lhcb/opg/-/issues/41 for the motivation for this MR. This MR is ready and tested.
cc @sponce @clemenci @bcouturi
~~**Note that is now part of LHCb!2993, so it should not be merged unless LHCb!2993 does not make it to production**~~RTA/2024.03.27Chris BurrChris Burrhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4010Consistently use RawBank::View as input to algorithms instead of RawEvent2024-03-24T20:56:03+01:00Gerhard RavenConsistently use RawBank::View as input to algorithms instead of RawEvent- remove use of `concat_alternatives`
- make lifetime-management `shared_ptr` in event store 'write only'
- remove 'stream' argument in make_ODIN -- use whatever stream is
defined for `make_raw` (i.e. `make_raw_banks`). This avoids the...- remove use of `concat_alternatives`
- make lifetime-management `shared_ptr` in event store 'write only'
- remove 'stream' argument in make_ODIN -- use whatever stream is
defined for `make_raw` (i.e. `make_raw_banks`). This avoids the need for
spurious, redundant `bind` statements downstream.
required for Rec!3351, Alignment!360 and Moore!2166Gerhard RavenGerhard Ravenhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4457remove obsolete `context()`2024-03-22T11:50:41+01:00Gerhard Ravenremove obsolete `context()``context()` stems from the time prior to the use of python to configure Gaudi applications, and was used to coherently switch sets of properties. At this point, at best it just interferes with the configuration, and creates confusion. He...`context()` stems from the time prior to the use of python to configure Gaudi applications, and was used to coherently switch sets of properties. At this point, at best it just interferes with the configuration, and creates confusion. Hence it is time to remove it all together so we can stop thinking/worrying about its effects.
requires Rec!3783, Boole!549Andre GuntherAnfeng LiAndre Guntherhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4480set the reserved bits to 31 for hlt raw banks2024-03-21T18:04:20+01:00Sevda Esenset the reserved bits to 31 for hlt raw banksClosing #221
@rmatev @gravenClosing #221
@rmatev @gravenhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4399Remove DAQKernel package (and migrate its users, eg. `bankKiller` and `RawEve...2024-03-19T15:44:31+01:00Gerhard RavenRemove DAQKernel package (and migrate its users, eg. `bankKiller` and `RawEventDump`)Retire Decoder::ToolBase and Decoder::AlgBase which were helpers to search for `RawBank`s from a search path of `RawEvent`s.
- migrate `bankKiller` and `RawEventDump` from `Decoder::AlgBase` to `Gaudi::Functional::MergingConsumer`
-...Retire Decoder::ToolBase and Decoder::AlgBase which were helpers to search for `RawBank`s from a search path of `RawEvent`s.
- migrate `bankKiller` and `RawEventDump` from `Decoder::AlgBase` to `Gaudi::Functional::MergingConsumer`
- migrate `IMuonRawBuffer` and `MuonRawBuffer` to Rec into the sole component source directory which needs them
- remove dictionary entry for `IMuonRawBuffer` (as that interface should be deprecated -- see #338)
- move the search code for `RawBank` which used to be in `Decoder::CommonBase` to the one bit of code remaining which used it, namely `ODINDecodeTool` (which itself may be obsolete/deprecated)
- remove the entire `DAQKernel` package, as its only raison d'etre was to host `Decoder::ToolBase` and `Decoder::AlgBase`.
must be used in conjunction with Rec!3698 which adds `IMuonRawBuffer` and `MuonRawBuffer` to Rec, and Allen!1386 which removes mention of DAQKernel from its dependencies (Allen didn't actually depend on DAQKernel). Same for MooreOnline!308. The change in printout in bankKiller requires Boole!531 which updates the references specifically for that change.https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4335Replace std::random_shuffle with std::shuffle2024-03-19T15:44:31+01:00Marco Clemencicmarco.clemencic@cern.chReplace std::random_shuffle with std::shuffle`std::random_shuffle` was deprecated in C++14 and (in principle) removed in C++17
Note: requires a change in the reference files in Boole and Rec`std::random_shuffle` was deprecated in C++14 and (in principle) removed in C++17
Note: requires a change in the reference files in Boole and Rechttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3510Turn off DetDesc configuration when using DD4hep2024-03-18T17:07:56+01:00Marco Clemencicmarco.clemencic@cern.chTurn off DetDesc configuration when using DD4hepThis MR completely hides the DetDesc configuration and databases when in DD4hep mode, so that we can check that everything has been migrated. As a corollary, this is expected not to change anything in the normal platforms, but increase t...This MR completely hides the DetDesc configuration and databases when in DD4hep mode, so that we can check that everything has been migrated. As a corollary, this is expected not to change anything in the normal platforms, but increase the number of failures in `+dd4hep` platforms.
I fixed as many tests as I could. Some tests still use DetDesc as they are meant to compare the two geometries while others are disabled as they cannot exist without DetDesc.
As discussed in https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/3461#note_5450094 and lhcb-datapkg/ParamFiles!47, `ParticleTable.txt` needs some more care as we are not going to take it from DetDesc anymore. For the time being I introduced a couple of copies of it to be able to run the tests (one could be removed with some massaging of the reference file).
Note also that 3 of the commits in this MR are minor fixes and could be merged independently.
/cc @jonrob @sponce @bcouturi @raaij @gligorov @rmatev @gravenChristopher Rob Jonesjonesc@hep.phy.cam.ac.ukChristopher Rob Jonesjonesc@hep.phy.cam.ac.ukhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4395Fix Allen UT geometry2024-03-11T23:28:06+01:00Jiahui ZhuoFix Allen UT geometryAddressing the issue: https://gitlab.cern.ch/lhcb/Allen/-/issues/467
This patch will resolve the Allen UT decoding issue for the 2024-expected MC by correcting the UT geometry dumping. As a result, HLT1 VeloUT will become operational on...Addressing the issue: https://gitlab.cern.ch/lhcb/Allen/-/issues/467
This patch will resolve the Allen UT decoding issue for the 2024-expected MC by correcting the UT geometry dumping. As a result, HLT1 VeloUT will become operational once again.
Test with: https://gitlab.cern.ch/lhcb/Allen/-/merge_requests/1407
FYI: @dovombru @bkhanji @ascarabo @hawu @bjashal @adeoyangJiahui ZhuoJoshua Ethan HorswillJiahui Zhuohttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4269UThit position methods2024-03-07T10:07:38+01:00Hangyi WuUThit position methodsWork with ~~https://gitlab.cern.ch/lhcb/Detector/-/merge_requests/438~~, https://gitlab.cern.ch/lhcb/Rec/-/merge_requests/3559 and https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2622Work with ~~https://gitlab.cern.ch/lhcb/Detector/-/merge_requests/438~~, https://gitlab.cern.ch/lhcb/Rec/-/merge_requests/3559 and https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2622Andre GuntherAndre Guntherhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4192Add ODIN EventType bit accessor and clean up TriggerTypes2024-03-07T10:07:38+01:00Rosen MatevAdd ODIN EventType bit accessor and clean up TriggerTypes1. Add `eventTypeBit(EventTypes mask)` accessor. For example, `eventTypeBit(ODIN::EventTypes::Lumi)` returns `true` if the `Lumi` event type bit is set and `false` otherwise.
2. Add a corresponding `setEventTypeBit(EventTypes mask, bool ...1. Add `eventTypeBit(EventTypes mask)` accessor. For example, `eventTypeBit(ODIN::EventTypes::Lumi)` returns `true` if the `Lumi` event type bit is set and `false` otherwise.
2. Add a corresponding `setEventTypeBit(EventTypes mask, bool value)` setter and remove the misleading `setEventType(EventTypes)`.
3. Remove unused or wrong (as of Run 3) `TriggerTypes`. Kept the still used `LumiTrigger` type and corrected `CalibrationTrigger` to the present value of `10`.
4. Rename obsolete (as of Run 3) `EventTypes` such as `TriggerMaskPhysics` to generic names such as `et_bit_10`. The rationale is that these bits are since recently used to identify particular bunches in the filling scheme, according to [1]. We deliberately do not name them explicitly (e.g. `TralingBunch`), because this information is redundant w.r.t. the filling scheme already in the condition database and therefore we don't want to commit to the long term meaning of said event type bits.
Notes:
5. `isFlagging` does not seem to be used in Run 3 so far. What is it's point and can it be re-used?
6. Recently someone noticed that checking `calibrationType() == CalibrationTypes::A` (`==0`) is not enough to select "Calib A" events. This is because when a even is not a Calib event, `0` is put in the `calibrationType()` bits (only 2). Currently the only thing one can do is also check for `triggerType() = TriggerTypes::CalibrationTrigger`. A cleaner solution (?) would be to have an event type bit to identify calibration triggers.
[1] https://indico.cern.ch/event/1265430/contributions/5457786/attachments/2669219/4626471/230620_LHCbRunMeeting_TFC.pdf
Needs lhcb/Rec!3606
/cc @falessio @edallocc @gcavalle @clemenci @raaijRosen MatevRosen Matevhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4442Simplification/Streamlining of GaudiConf.reading2024-03-06T17:00:32+01:00Gerhard RavenSimplification/Streamlining of GaudiConf.readingThere are only two functions in GaudiConf.reading which are (need to be) exported: `do_unpacking` and `type_map`. A few other functions are only invoked by reco_objects_for_spruce.py, and are moved there. `do_unpacking` is a special case...There are only two functions in GaudiConf.reading which are (need to be) exported: `do_unpacking` and `type_map`. A few other functions are only invoked by reco_objects_for_spruce.py, and are moved there. `do_unpacking` is a special case (it configures unpacking using configurables for GaudiPython, bypassing PyConf) and it is only used by qmtests of the sprucing. This MR streamlines and simplifies `do_unpacking` so that it exactly meets the requirements of that use case, and removes any suggestion of it being more general, which it actually isn't.
Must be applied in conjunction with Moore!2983Sebastien PonceSebastien Poncehttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4452ProtoParticle: avoid comparing SmartRef to 0, use nullptr instead2024-03-01T11:00:34+01:00Gerhard RavenProtoParticle: avoid comparing SmartRef to 0, use nullptr insteadAndre GuntherJoshua Ethan HorswillAndre Guntherhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4426FPE Auditor: Issue warning not exception when trapping support unavailable2024-02-16T10:56:28+01:00Christopher Rob Jonesjonesc@hep.phy.cam.ac.ukFPE Auditor: Issue warning not exception when trapping support unavailableHopefully allows tests to run on arm (armv8.1_a-el9-gcc13-opt), with FPEAuditor just doing nothing other than issuing a warning.
Should hopefully address some (RICH) tests in LHCb and Rec that are currently failing.Hopefully allows tests to run on arm (armv8.1_a-el9-gcc13-opt), with FPEAuditor just doing nothing other than issuing a warning.
Should hopefully address some (RICH) tests in LHCb and Rec that are currently failing.https://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4449Fixed Muon decoding test2024-02-14T22:17:09+01:00Sebastien PonceFixed Muon decoding testhttps://gitlab.cern.ch/lhcb/LHCb/-/merge_requests/4227Fix bug in VP encoding and decoding of topologyID2024-02-14T18:48:29+01:00Daniele Passarodaniele.passaro@cern.chFix bug in VP encoding and decoding of topologyID### Fixed bugs in the encoding and the decoding of the VPRetinaCluster word, both for light-clusters and full-clusters.
In the current version of the code, clusters with the wrong pixel-list were created and decoded. Clusters with empty ...### Fixed bugs in the encoding and the decoding of the VPRetinaCluster word, both for light-clusters and full-clusters.
In the current version of the code, clusters with the wrong pixel-list were created and decoded. Clusters with empty pixel-list were also created.\
The latter case was the cause for the Floating Point exceptions reported in https://gitlab.cern.ch/lhcb/LHCb/-/issues/315 and https://gitlab.cern.ch/lhcb/Lbcom/-/issues/10.
A reference update is also needed.\
Indeed the reference MC files were created with the wrong topology encoding. The corrected algorithms might misbehave with the old topology version, resulting in a slightly lower tracking efficiencies (~0.1%) if evaluated on those MC simulations using MC checking.
The real tracking efficiencies are not affected, since the tracking sequences do not use the VPRetinaCluster topologyID information. As result, no porting in Allen is required as topologyID is not used in Allen sequences.
FYI: @flazzari
This MR depends on https://gitlab.cern.ch/lhcb/Moore/-/merge_requests/2869 .Rosen MatevDaniele Passarodaniele.passaro@cern.chRosen Matev