Re:Can we trust floating-point numbers? (2006) [pdf]
by stephencanon on Tuesday September 16, 2014 @04:27PM
(#8326905)
My understanding is that Excel mostly uses double, but with a bunch of hacks bolted on to make some results “seem” right. If anything this makes the situation worse. IIRC, for example, it’s possible to construct numbers such that any desired subset of the following three properties do or do not hold:
It’s been ~10 years since I really poked around at Excel numerics, so my recollection may not be perfect, but there were definitely a number of hacks beyond straightforward double-precision arithmetic when I looked at it.
[ Parent
| on HN ]
There are no comments on this comment yet.