Skip to content

[surface model] Mixed precision improvements

Severin Diederichs requested to merge mixed_precision_improvements into surface_model

This MR does a few improvements to the mixed precision handling:

  • it adds the handling of the ConeTolerances in mixed precision. Note that it is not stable to have the ConeTolerance being the same as the kTolerance. We should aim to reduce the kTolerance to lower values in float, to be tested.
  • a relative tolerance is added (but not much used yet).
  • MakePlusTolerant(0) = tolerance
  • a Tolerance() implementation was removed as it was the same as kToleranceDist<>.
  • Some signs in the RingMask were switched.

Merge request reports