Skip to content

WIP: Let's discuss single-precision tolerances

Hadrien Benjamin Grasland requested to merge hgraslan/acts-core:f32-fixes into master

This MR is a preliminary output of my study on the single-precision portability of ACTS. It should probably not be merged as is, but is only meant as a strawman to get a discussion started.

It builds on top of !490 (merged), so you may want to explore it commit by commit and/or diff it locally against that branch in order to get a clean diff.

In this branch, I've been running the ACTS test suite under Verrou's single-precision emulation mode, and tuning up test tolerances until the test suite mostly passes. At this point, the test suite does not fully pass because it hits the same off-diagonal covariance term imprecision issues as !490 (merged), but it gets close enough to get the discussion started. => That problem is solved (also, found some room for doing with narrower tolerances, under investigation)

Here is my question before I go further in my single-precision studies: which of these tolerance changes would be acceptable (in the sense that we would still produce precise enough physics), and which wouldn't?

Depending on the answer to this question, I will next investigate specific areas of unacceptable imprecision, in order to figure out whether the relevant parts of ACTS should stay in double precision, or have a "precision bottleneck" somewhere that can easily be resolved to make single precision viable.

Edited by Hadrien Benjamin Grasland

Merge request reports