AthenaServices + AthenaPoolExampleAlgorithms: Prepare ThinningCache for setting lossy float compression information
This MR builds on !35326 (merged). It extends Scott's recent updates on thinning (to accommodate per-stream variable selection in AthenaMT
) to use the same infrastructure for setting/getting the lossy float compression information.
The main idea here is to store the lossy float compression information (which is essentially a map that holds a set of AuxIDs
that should be compressed for each compression level) into the ThinningCache
as opposed to the EDM objects themselves. When fully implemented, AthenaOutputStream
is going to store the SG::CompressionInfo
into the event store. ThinningCacheTool
inserts this into a ThinningCache
and one can query the cache via ThinningInfo
. The latter is actually passed onto SG::copyAuxStoreThinned
where the thinning/selection/compression is applied.
This MR puts everything into place for filling the ThinningCache
for the lossy float compression information. However, it's not yet used (i.e. the last bit above), which will be implemented in an upcoming MR. Also, once we switch to this new method, we won't need the IAuxStoreCompression
that is implemented in AuxContainerBase
anymore. So, that bit will also be cleaned up.
This MR is self-consistent and it's already fairly hefty. So, in order to ease the review, I'm submitting this now and will complete the development in a future MR, which will also include the clean-up (but I might also break that bit down too).
Merge request reports
Activity
added full-unit-tests label
WARNING: big files (>100K) are found in the changeset 144K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_ReWriteAgain.ref 140K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_ReWrite.ref 116K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_Concat.refadded Core Database master review-pending-level-1 labels
CI Result FAILURE (hash f7975931)Athena AthSimulation AthGeneration AnalysisBase externals cmake make required tests optional tests Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 18423]mentioned in merge request !35326 (merged)
I'll take a closer look at the unit test results on Monday, but from a quick scan, I see almost all of the failures are the same as the recent nightlies. There are new ones, e.g.
CaloRec_CaloThinCellsByClusterAlg_test_ctest
, but I have a hunch that at least some of these might be related to !35389 (merged) which got merged earlier today since the pattern looks suspiciously related:-Py:PropertyProxy WARNING Found 2 duplicates among the 1069 genConfDb files : -Py:PropertyProxy WARNING -------------------------------------------------- -Py:PropertyProxy WARNING -<component name>: <module> - [ <duplicates> ] -Py:PropertyProxy WARNING -------------------------------------------------- -Py:PropertyProxy WARNING -BSFilter: OverlayByteStreamAlgs.OverlayByteStreamAlgsConf - ['OverlayByteStreamUtils.OverlayByteStreamUtilsConf', 'OverlayByteStreamUtils.OverlayByteStreamUtilsConf'] -Py:PropertyProxy WARNING -ByteStreamMultipleOutputStreamCopyTool: OverlayByteStreamAlgs.OverlayByteStreamAlgsConf - ['OverlayByteStreamUtils.OverlayByteStreamUtilsConf', 'OverlayByteStreamUtils.OverlayByteStreamUtilsConf'] -Py:PropertyProxy WARNING Fix your cmt/requirements file !!
So let me tag @tadej as these will probably fail in today's nightly, too.
added review-approved label and removed review-pending-level-1 label
WARNING: big files (>100K) are found in the changeset 144K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_ReWriteAgain.ref 140K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_ReWrite.ref 116K in file Database/AthenaPOOL/AthenaPoolExample/AthenaPoolExampleAlgorithms/share/AthenaPoolExample_Concat.refadded review-pending-level-1 label and removed review-approved label
added review-approved label and removed review-pending-level-1 label
CI Result FAILURE (hash f7975931)Athena AthSimulation AthGeneration AnalysisBase externals cmake make required tests optional tests Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 18513]- Resolved by Vakhtang Tsulaia