xAODCore: FloatCompressor bugfix + documentation
In the current implementation, we seem to be missing an f in the mask value when we check if the original value is less than the allowed maximum. This'll be a potential problem once we're in the vicinity of the maximum, e.g.:
Value = 3.40282e+38 ( binary: 0 11111110 11111111111111111111111 )
Current = inf ( binary: 0 11111111 00000000000000000000000 )
Updated = 3.38953e+38 ( binary: 0 11111110 11111110000000000000000 )So, after the fix we'll be rounding down avoiding infinity.
I also added a bit of documentation to explain what the code does as far as I follow :)
It's interesting to note that we're overall very close to TensorFlows' "round a half value to the nearest even" that they use for bfloat16 (see here) except when the least significant bit is 0, the rounding bit is 1, and all the sticky bits are zeros. In this case, they round down since this is exactly at half and the result is already even, whereas we round up. For example:
Value = 3.97656 ( binary: 0 10000000 11111101000000000000000 )
Ours = 3.98438 ( binary: 0 10000000 11111110000000000000000 )
Theirs = 3.96875 ( binary: 0 10000000 11111100000000000000000 ) I'm not 100% where our logic is coming from exactly apart from the fact that it's based on AthenaPoolCnvSvc/Compressor that seems to date back to 2007 :)
-s