MC-matching fixes for 2023 MC central productions

Moore v54r16p4 was released for running HLT2 centrally on 2022 and 2023 MC productions. However, some fixes are needed on top for MC matching. They are all already included in master but need to be ported to 2023-hlt2pp-patches

• use Retina full clusters for mc-matching. MR open in !3154 (merged)
• truth matching fix for persistreco, added to master in !2781 (merged) Ported to 2023-hlt2pp-patches in !3222 (merged)
• truth matching fix in persistency, mostly affecting neutral objects. Added to master in LHCb!4453 (merged) Ported to 2023-hlt2pp-patches in LHCb!4498 (merged)

FYI @cagapopo @sesen @kreps @decianm

Thanks to @lugrazet for reporting.

Additionally LHCb!4524 (merged) is needed for missing public headers in detdesc builds.

changed milestone to %RTA/2024.04.04-MC2023

added Simulation hlt2 labels

assigned to @sesen and @cmarinbe

changed the description

mentioned in merge request !3222 (merged)

mentioned in merge request LHCb!4498 (merged)

changed the description

mentioned in merge request lhcb-datapkg/AnalysisProductions!1041 (merged)

Hi all, but especially @sesen, I'll report here.

As discussed previously I wanted to verify that these three changes have the effect we expect and 'fix' MCDecayTrees.

Short Answer: It doesn't seem to be enough. Although the changes listed above bring a significant improvement in the fraction of MCDecayTree (~33% -> ~70%) events being stored there's still ~30% of the events not being included.

Maybe there's another bug fix that isn't being ported into 2023-hlt2pp-patches or is an issue that is currently undiagnosed in master itself. I am not too sure either way unfortunately.

Further Elaboration

Finally got round to testing the Hlt2->DaVinci with each of the changes listed above and I'm still not seeing the expected behaviour.

Did as follows: Ran Hlt1 over 'actual' 2023 Z MC with: v54r12p1 i.e. 2023-hlt1pp-patches/latest via some representative Hlt1 script with flagging mode over 100 events [ew_projects_run3/hlt1]

Built a stack including all of !3154 (merged) + !3222 (merged) + LHCb!4498 (merged) above 2023-hlt2pp-patches (see: make report below)

Ran representative Hlt2 script [ew_projects/hlt2] with flagging mode over the Hlt2 output.

Then ran DaVinci for a Z MCDecayTree and EventTuple (via: mcdt_tuple.py + tuple_input.yaml).
I expect in a completely functional MCDecayTree to see 100% of the candidates making it into the MCDecayTree, instead we only see 69 Z candidates out of 100 events making it into the MCDecayTree (while the EventTuple is completely filled with 100/100 events).

I performed this without the Hlt2 passthrough line included, and we see 33 MCDecayTree and 45 events total.

I performed this without the Hlt1 stage, i.e. just Hlt2 on a digi file, and we see 69 MCDecayTree and 100 events total, i.e. completely the same as when we include Hlt1 flagging stage.

Not sure what is causing this problem.

make report

I took these from the RTA stack that was released for 2023-hltX-patches and verified that each project's 2023-patches branch (or similar) had no changes included since then so I was free to use the release version.

INFO     Detector         is at v1r19p1 (12234d74eb)
INFO     Gaudi            is at v36r16 (ea80daf860)
INFO     LHCb             is at origin/sevda-truth-matching-issue-738 (3b8bedafcf)
INFO     Lbcom            is at v34r15p2 (5310c97c9c)
INFO     Rec              is at v35r15p2 (b5d50d47f6)
INFO     Allen            is at v3r16p2 (1946d2c4e0)
INFO     Moore            (5d7cd6d7fc, HEAD -> lugrazet-2023patches-truthmatching) is 8 commits ahead origin/sevda-truth-matching-issue-738 (bf8114cfc5)
INFO     DBASE/AppConfig  is at origin/master (b19e99385b)
WARNING  DBASE/PRConfig   (d407432090, HEAD -> master) is 4 commits behind origin/master (271780ddbb)
INFO     PARAM/ParamFiles is at origin/master (b437f945ef)
WARNING  utils            (b05e9ff7d7, HEAD -> master) is 3 commits behind origin/master (29f936e87c)

@cagapopo These 3 MRs are ready. Detdesc build has issues but not related to any of the open MRs. How do you want to proceed?

changed the description

@jonrob The four MRs are ready to finalize 2023-hlt2pp-patches before a release: !3154 (merged) !3222 (merged) LHCb!4498 (merged) LHCb!4524 (merged) Nightlies look good to me with expected changes.

thanks @jonrob

@cmarinbe you can close this issue.

thanks a lot both! Closing.

marked the checklist item use Retina full clusters for mc-matching. MR open in !3154 (merged) as completed

marked the checklist item truth matching fix for persistreco, added to master in !2781 (merged) Ported to 2023-hlt2pp-patches in !3222 (merged) as completed

marked the checklist item truth matching fix in persistency, mostly affecting neutral objects. Added to master in LHCb!4453 (merged) Ported to 2023-hlt2pp-patches in LHCb!4498 (merged) as completed

closed

mentioned in merge request lhcb-datapkg/AnalysisProductions!1059 (merged)