B2OC: Add 30 new lines, rate reduction and minor fixes

This MR containes several development from B2OC PAWG since last time (!1199 (merged)), including the following MR:

New lines

• 16 Omegab and Xib lines by Mengzhen Wang (were previously B&Q Stripping lines), 8 Hlt2 and 6 Sprucing: !1229 (merged)
• 14 Lambdab lines by Shunan Zhang, all in Hlt2: !1344 (merged)

Work towards DPA task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/78.

Rate reduction

• Tuning B -> Dshhh lines by Philippe D'Argent: !1341 (merged)
• Tuning EM lines by Alessandro Bertolin: !1347 (merged)
• Tuning B(c)+ -> DD lines by Fionn Bishop: !1351 (merged)
• Tuning B0 -> D0hh, D0 -> HH(HH) lines by Aidan Wiederhold: !1363 (merged)
• Tuning B -> DDKpi lines by Mark Whitehead: !1364 (merged)
• Tuning B -> Dh, Dhh, Dhhh, DstMuNi lines by Martin Tat and Seophine Stanislaus: !1371 (merged)
• Tuning Bc lines by Fernanda Abrantes: !1384 (merged) and !1409 (merged)
• Moving Hlt2 lines with high rates (> 300 Hz) to Sprucing by Alessandro Bertolin: !1409 (merged)

additional details can be found in this presentation: https://indico.cern.ch/event/1118176/contributions/4695390/attachments/2402878

the retention table of all Hlt2 lines included is in attachment b2oc-hlt2-minbias-220305.txt

New features

• Add a monotone Lipschitz neural network MVA filter by Fionn Bishop and Alessandro Bertolin: !1345 (merged)
• Add flavor tagging for Hlt2 and Sprucing by Alessandro Bertolin: !1319 (merged) and !1324 (merged)

Requires lhcb-datapkg/ParamFiles!41 (merged)

Fixes

• Name fixes by Alessandro Bertolin: !1376 (merged)
• PID cut fixes by Alessandro Bertolin: !1382 (merged) and !1387 (merged)
• Fix cuts in KS builder by Martin Tat: !1383 (merged)
• Fix a particle builder by Alessandro Bertolin: !1392 (merged)
• Fix importing bug in spruce_b2oc.py by Alessandro Bertolin: !1393 (merged)
• Change path of the json file used by MVA by Alessandro Bertolin: !1417 (merged)
• Comment out lines with neutral and DD particles by Shunan Zhang: !1420 (merged)
• Fix Spruce tests by Alessandro Bertolin: !1422 (merged)
• Align DsK and DsK LTU selections, fix path of SigmaNet json file: !1423 (merged)
• Remove check_overlap function: !1426 (merged)
• Move one Sprucing line from FT dictionary to default: !1427 (merged)

assigned to @abertoli

assigned to @shunan

requested review from @mvesteri

requested review from @poluekt

added B2OC DPA-WP3 all-slots hlt2 labels

changed the description

added RTA label

changed the description

changed the description

mentioned in merge request lhcb-datapkg/ParamFiles!41 (merged)

Hi @abertoli, can I run test or do you want to have a full review by @mvesteri and @poluekt at first?

added 2 commits

• 4d7965ef - B2OC: change path and name for the .json file of the B2OC generic B-hadron NN
• 2081b7dc - Merge branch 'b2oc-abertoli-sigmanetpath' into 'b2oc_upgrade'

Compare with previous version

added ci-test-triggered label

• [2022-03-08 15:20] Validation started with lhcb-master-mr#3904
• [2022-03-09 00:08] Validation started with lhcb-dd4hep#1066
• [2022-03-09 00:10] Validation started with lhcb-gaudi-head#3182
• [2022-03-09 00:24] Validation started with lhcb-head-2#492
• [2022-03-09 00:27] Validation started with lhcb-head#3204
• [2022-03-09 00:31] Validation started with lhcb-lcg-dev3#1929
• [2022-03-09 00:37] Validation started with lhcb-lcg-dev4#1946
• [2022-03-09 01:07] Validation started with lhcb-head-2#492
• [2022-03-09 01:09] Validation started with lhcb-run3-cleanup#457

@abertoli Is this MR really supposed to be added to all-slots?

removed all-slots label

removed DPA-WP3 label

added DPA-WP1 selection labels

changed title from B2OC: Hlt2 lines rate reduction, lines with rate > 300 Hz moved to Spruce to B2OC: Add 30 new lines, rate reduction and minor fixes

changed the description

@nskidmor (cc @shunan) the stripping line BdToDsmK_DsmToHHH has been replaced by BdToDsmK_DsmToKpKmPim

hence the failure in attachment, I guess the only solution is to fix the line name being used in the test in THIS MR, do you agree ?

@nskidmor (cc @shunan) can you have a look to the screen picture in attachment ? I have 2 points

1. the most trivial one: this is a B2OC MR so what can we do for the failures related to RD lines ? we can not change them in this MR ..

2. let's take the first B2OC line that gives the failure: B2OC_BdToD0D0_D0ToHHOrHHHH, it has in the final state 8, 6 or 4 tracks .. may be the problem comes from the candidates with 8 tracks .. but is the problem that the amount of bytes needed to save 8 tracks is too large ? or that there are too many candidates with 8 tracks and then the number of bytes is too large ?

this one: B2OC_BdToD0KPi_D0ToHH has only 4 tracks and gives the error anyway

what can we do for this ?

@msaur (cc @shunan) any clue about the meaning of the attached snapshot ?

is it like try again any you could be more lucky ?

mentioned in merge request LHCb!3268 (merged)

added 2 commits

• f36cf94f - B2OC: comment out lines with neutral or DD particles
• b87209e6 - Merge branch 'b2oc-shunan-comment-lines' into 'b2oc_upgrade'

Compare with previous version

changed the description

added 2 commits

• 21abcd09 - B2OC: fix for the Spruce tests (shifters PLEASE ignore)
• b33bb4c7 - Merge branch 'b2oc-abertoli-sprucetestsfix' into 'b2oc_upgrade'

Compare with previous version

changed the description

@msaur (cc @shunan) in collaboration with @nskidmor I think I have fixed the Spruce tests with !1422 (merged)

what is pending is why Hlt2B2OC_B_SigmaNet_Run3.json is not found

resolved all threads

added 2 commits

• 91e96581 - B2OC: align DsK and DsK LTU selections, fix path of SigmaNet json file
• ee2e1446 - Merge branch 'b2oc-abertoli-fixes' into 'b2oc_upgrade'

Compare with previous version

changed the description

added 2 commits

• 8cf6631f - B2OC: remove check_overlap function
• 470c2464 - Merge branch 'b2oc-shunan-remove-checks' into 'b2oc_upgrade'

Compare with previous version

@msaur (cc @shunan) we have implemented the fix suggested by @rmatev for the Hlt2B2OC_B_SigmaNet_Run3.json file not being found (I have tested it locally on lxplus but also the previous syntax was fine on lxplus)

if this also fails may be we should consider to merge lhcb-datapkg/ParamFiles!41 (merged) first and then ran the ci because in ParamFiles we are adding a .json file used only by B2OC code ... there is no new code being ran in ParamFiles ...

but fingers crossed

• [2022-03-10 09:33] Validation started with lhcb-master-mr#3926
• [2022-03-10 16:50] Validation started with lhcb-master-mr#3935

@msaur (cc @shunan @nskidmor) there are big improvements on the tests ! just one failure .. see screen capture below

may be we missed this value yesterday when we increased the alarm level to 80000

any additional suggestion from @nskidmor is welcome

added 2 commits

• b2de780b - B2OC: remove BdToDmKsLLPi_DmToHHH from FT Spruce lines dictionaly, book it as default line
• ae680ba9 - Merge branch 'b2oc-abertoli-fix' into 'b2oc_upgrade'

Compare with previous version

resolved all threads

@msaur (cc @shunan @nskidmor) I have just taken out all flavor tagging data for the line BdToDmKsLLPi_DmToHHH, if we have understood what's going on the Spruce error should go

@msaur can you please test again .. I hope for the last time ..

@nskidmor you are lucky to have those silly B2OC people doing all sort of nasty things that trigger your errors and warnings :-) :-) :-)

changed the description

@mvesteri you wrote above you are happy with this and now the relevant ci tests look fine

@poluekt do you have any feedback for us ?

can we pass this on to Rosen ?

assigned to @rmatev

resolved all threads

approved this merge request

added new feature label

merged

mentioned in commit 909a88f1

mentioned in issue #411 (closed)