Re:Formatting floating point numbers
by cbsmith on Saturday June 15, 2019 @02:19AM
(#20188647)
You'd be surprised. Decades ago I had to convert mortgage software that was using 32-bit floating point to fixed point arithmetic because they discovered that the cumulative errors from computing payments over 30+ year mortgages could lead to non-trivial inconsistencies.
Don't knock the impact of cumulative loss of precision.
Don't knock the impact of cumulative loss of precision.