Skip to content

Optimise basket sizes created by RootCnvSvc

Chris Burr requested to merge lhcb/Gaudi:cburr/better-basket-sizing into master

Previously the basket size configuration could only be coarsely applied to all branches but in most real world files the size of the data in each branch varies significantly. Setting a small value causes large branches to be packed into individual baskets and prevents effective compression. Setting a large value improves the compression ratio of large branches but causes excessive memory usage.

Instead we change the parameters to only specifiy the maximum and minimum branch size and provide a target for the number of events to pack into each basket. We then use the first event to estimate the size of the branch.

This makes the branching very sensitive to the first event size however this seems to work okay in practice. If needed the basket sizes could be refined after several events to give a more precise estimate.

Edited by Chris Burr

Merge request reports

Loading