Skip to content

xAODCore : Configure the float compression rounding based on the input mantissa bits

Hi guys,

@elmsheus found out a peculiarity in the current implementation of the xAOD::FloatCompressor. In short, the offset that is being applied to remove the bias from truncation is hardcoded with the assumption that we always use 7 bits mantissa (i.e float to short compression w/ sign/exponent bits intact). However, this might not be true since the user can configure it independently. Therefore, this MR updates the rounding to be aware of the mantissa bits that the user asked for. I see we already have AthenaPoolCnvSvc/Compressor which seems to get this right. So the solution is largely based on what's being done there. I'd appreciate a fresh pair of eyes to double check to make sure I'm not missing anything. Many thanks.

Best, Serhan

@ssnyder @akraszna @jcatmore

Merge request reports