Re:9999999999999999.0 – 9999999999999998.0
by dbaupp on Sunday January 06, 2019 @01:36AM
(#18836940)
I believe IEEE754 floats have that subtraction/addition guarantee (as long as the hardware doesn't map subnormals to zero). The problem in this case is the input numbers are rounded when they are converted from text/decimal to a float, and so aren't exact.
Re:9999999999999999.0 – 9999999999999998.0
by int_19h on Sunday January 06, 2019 @09:00PM
(#18842165)
> I believe IEEE754 floats have that subtraction/addition guarantee (as long as the hardware doesn't map subnormals to zero).
They don't - all 11 bits of the exponent (for float64) are in use, so you can have something like 1e300, and then you can't e.g. add 1 to it and get a different number.
>>> x = 1e100
>>> x
1e+100
>>> y = x + 1
>>> y
1e+100
>>> x - y
0.0
Re:9999999999999999.0 – 9999999999999998.0
by dbaupp on Tuesday January 08, 2019 @04:34PM
(#18859944)
That's something else: x == y, so the difference must be 0. What you said above is
if x != y, x - y != 0
if dx != 0, x + dx != x
The only example off the top of my head that is floating point is C# "decimal", which actually originates from the Decimal data type in OLE Automation object model (which could be seen in VB6, and can still be seen in VBA):
https://msdn.microsoft.com/en-us/library/cc237603.aspx
Note this bit:
"scale: MUST be the power of 10 by which to divide the 96-bit integer represented by Hi32 * 2^64 + Lo64. The value MUST be in the range of 0 to 28, inclusive."
The reason why it's limited to 28 is because the 96-bit mantissa can represent up to 28 decimal digits exactly. The way it's enforced, any operation that produces a result outside of this range is an overflow error (exception in .NET).