DerivationFrameworkPhys: Add postIncludes for testing Lossy Float Compression in DAOD_PHYS/DAOD_PHYSVAL production
This MR adds four postInclude
files that'll ease testing lossy float compression during DAOD_PHYS
/DAOD_PHYSVAL
production. Each postInclude
file is self-sufficient, defining a list of containers for which the lossy float compression should be applied, and configures the underlying stream appropriately. The usage is simply:
--postInclude 'AODtoDAOD:DerivationFrameworkPhys/postInclude.PHYS_LowLossyFloatCompression.py'
The compression lists are preliminary for testing (and identical between the files). They are simply built by looking at the top disk users in the DAOD_PHYS
files in the most recently master nightlies in local tests. Needless to say, these can/will be refined as we go along.
For each derivation type, these files can be used in conjunction (i.e. included simultaneously), although at the moment this won't do anything more than simply adding the high compression one alone (because the lists are identical and the high compression has precedence).
In local tests, the low compression seems to reduce the DAOD_PHYS
size by about 10% and the high compression by about 20% (500 events over a data file).
The eventual plan is to add the relevant configuration bits to DerivationFrameworkCore/SlimmingHelper.py
so that the postInclude
hooks will not be needed in production.