Re:Demystifying Floating Point Precision
by evincarofautumn on Wednesday November 22, 2017 @10:13PM
(#15762469)
Doing calculations with an exact number type such as [big] rationals, and then explicitly converting to floating point or rounding/truncating digits of precision as needed, is a pretty good middle ground for many applications. 109/100 − 109375/100000 = 109/100 − 35/32 = −3/800 = −3.75ᴇ−3.
[ Parent
| on HN ]
There are no comments on this comment yet.