add optional pyconf NTupleSvc basket size, requires gaudi/Gaudi!1648

Customise the Branch BasketSize for FunTuple creation to reduce 'many-trees' in a single Analysis production memory consumption.

Requires LHCbIntegrationTests!84 (merged) and Gaudi v38r1p1 (see gaudi/Gaudi!1648 (merged) and lhcb-core/LHCbNightlyConf!1263 (merged))

FYI : @cburr , @clemenci , @sponce

mentioned in merge request gaudi/Gaudi!1648 (merged)

added RTA label

added 1 commit

• 0fb16c9d - Fixed formatting

Compare with previous version

added 1 commit

• 77bd5271 - add ntuple_basketsize in LbExec

Compare with previous version

resolved all threads

approved this merge request

added 5 commits

• 77bd5271...6a0a166e - 2 commits from branch 2024-patches
• f7c512a6 - add optional pyconf basket size
• b991c27b - Fixed formatting
• f875a86b - add ntuple_basketsize in LbExec

Compare with previous version

added 1 commit

• b9c304d3 - Fixed formatting

Compare with previous version

added DPA-WP3 label

marked this merge request as ready

@cburr , @masmith , @sponce , @clemenci what do you suggest to do in order to have this and the change in Gaudi included for DaVinci v64r11 release coming soon?

• i think Gaudi target v38r1-patches which means it's exactly as v38r1 except for 2 line change, it sounds like something very simple to 'back-port' in a clean way?

I configured locally the code from the AP in lhcb-datapkg/AnalysisProductions!1670 (merged)

Here the comparison of running the nominal DaVinci/v64r10 released vs the one compiled locally and configuring in the yaml of lbexec ntuple_basketsize=1024

Improvement is significant ( i think ) and will probably keep my AP below the 5-6 Gb limit of processing on the grid.

in my local job after a while i get some errors tough ...

I guess i would need to set a bigger basket size or tune it a bit

/ci-test --model lhcb-2024-patches gaudi/Gaudi!1648 (merged)

added ci-test-triggered label

• [2024-10-14 14:35] Validation started with lhcb-2024-patches-mr#1647
• [2024-10-15 10:25] Validation started with lhcb-2024-patches-mr#1656
• [2024-10-21 07:53] Validation started with lhcb-2024-patches-mr#1687
• [2024-10-21 15:53] Validation started with lhcb-2024-patches-mr#1690

assigned to @zejia

mentioned in issue Moore#866 (closed)

added needs Gaudi release label

mentioned in merge request LHCbIntegrationTests!84 (merged)