> There are also 255 representations of almost all representable numbers. [...] Aside from the fact that you're wasting an entire byte of your representation
How is this different than any other floating-point representation? I'm pretty sure IEEE floating-point has the same redundancy, though numbers are normalized so comparisons are cheaper as you note. But IEEE doubles "waste" even more bits due to the 2^52 representations of NaN.
> For most comparisons [...] it will take around FIFTY INSTRUCTIONS TO CHECK IF TWO NUMBERS ARE EQUAL OR NOT.
Good point, sounds like a notable weakness and barrier to adoption.
> Crockfords bugaboo with IEEE 754 floating-point is bizarre, verging on pathological.
He calls it "the most frequently reported bug in JavaScript." Wouldn't you be interested in improving on the most common cause of user confusion in a technology you care about?
I personally think that the way to handle floating-point confusion is better user education. However, if you really want a decimal standard, then, as I mentioned above, there already is one that is part of the IEEE 754 standard. Not only do there exist hardware implementations, but there are also high-quality software implementations.
A better approach to making things more intuitive in all bases, not just base 10, is using rational numbers. The natural way is to use reduced paris of integers, but this is unfortunately quite prone to overflow. You can improve that by using reduced ratios of – guess what – floating point numbers.
You are not correct. The smallest significand possible is 1x10^1, but you can't delve further into positive exponents. Conversely, 56 signed bits allows the largest integer power of 10 as 10 000 000 000 000 000 so the exponent will be -15. So there are exactly 17 representations of 10, and that's the worst it gets. All other numbers except powers of 10 have fewer representations, and most real world data affected by noise has a single representation because they use the full precision of the significand, and you can't shift them to the right or left without overflow or loss of precision.
So the redundancy is much less than you think, one in 10 real values has two representations, one in 100 has three etc. This is common for other decimal formats and not that big of a problem, detecting zero is a simple NOR gate on all significant bits.
The real problem with this format is the very high price in hardware (changing the exponent requires recomputing the significand) and complete unsuitability for any kind of numerical problem or scientific number crunching. Because designing a floating point format takes numerical scientists and hardware designers, not assembly programmers and language designers.
Heck, the only reason he put the exponent in the lower byte and not the upper byte, where it would have ensured a perfect compatibility to most positive integers, is that X64 assembly does not allow direct access to the upper byte.
[1] http://en.wikipedia.org/wiki/Decimal128_floating-point_forma...
Nobody who does anything with numbers believes that! Even if all you can do is count your fingers you believe in the difference between integers and floats. They have different algebraic properties entirely and it takes a whole hell of a lot of work to get from one to the other---there's even a whole class (fractions) in between.
It's confusing because it is very difficult to look at a decimal number and know whether it can be represented exactly as base-2 floating point. It's especially confusing because you get no feedback about it! Here is a Ruby session:
irb(main):001:0> 0.1
=> 0.1
irb(main):002:0> 0.2
=> 0.2
irb(main):003:0> 0.3
=> 0.3
irb(main):004:0> 0.4
=> 0.4
irb(main):005:0> 0.5
=> 0.5
They all look exact, right? Well actually only 0.5 is (IIRC?). All the others are actually approximations internally.I couldn't find any (like not a single one), so I wrote a program myself to do it: http://blog.reverberate.org/2012/11/dumpfp-tool-to-inspect-f...
If you know of a program in any of these languages that will print this value for "0.1" using built-in functionality, please let me know because I would love to know about it.
Likewise the precise value of double(1e50) is 100000000000000007629769841091887003294964970946560. Anything else is an approximation of its true value.
In another message you said that what's really important is that the string representation uniquely identifies the precise value. While that will help you reconstruct the value later, it does not help you understand why 0.1 + 0.2 != 0.3.
>>> from decimal import Decimal
>>> Decimal(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
>>> Decimal(1e50)
Decimal('100000000000000007629769841091887003294964970946560') toRational 0.1 = 3602879701896397 % 36028797018963968
toRational 1e50 = 100000000000000007629769841091887003294964970946560 % 1
Both of those values are precise, although I admit the first isn't really useful for human beings. julia> big(0.1)
1.000000000000000055511151231257827021181583404541015625e-01
julia> big(1e50)
1.0000000000000000762976984109188700329496497094656e+50
Note that this is exactly why you don't normally want to construct BigFloats this way. Instead you want to do this: julia> BigFloat("0.1")
1.000000000000000000000000000000000000000000000000000000000000000000000000000002e-01
julia> BigFloat("1e50")
1e+50 rasky at monocle in ~
↪ python
Python 2.7.5 (default, Sep 2 2013, 05:24:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 0.1
0.1
>>> ^D
rasky at monocle in ~
↪ python3
Python 3.3.3 (default, Dec 24 2013, 13:54:32)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 0.1
0.1
It used to, but it was changed to reduce users' confusion. Python 3.3.5rc1 (default, Feb 23 2014, 17:44:29)
>>> 0.3 == 0.2 + 0.1
False
>>> 0.2 + 0.1
0.30000000000000004
>>> 0.3
0.3This isn't true. Only zero has 255 representations. All other numbers are limited by the size of the coefficient, 16 digits. So a number with 1 significant digit would have 16 representations, one with two would have 15, etc. Not that I'm defending the format, but if we're critiquing it for being sloppy, we need to be careful about such things.
> For most comparisons (no, comparing numbers with the same exponent is not the norm)
I'd say that exponent 0 is the norm, meaning precise numbers in a really wide range, isn't it?
What I meant by exponent zero not being the norm is that most numbers you work with won't have exponent zero. Specifically only one in 2^16 of the possibly representable values and only one in 2^24 of the possible representations. If you're using the "natural" representation, then only the integers -9 through 9 have zero exponent.
"Because the significand is not normalized, most values with less than 16 significant digits have multiple possible representations; 1×102=0.1×103=0.01×104, etc. Zero has 768 possible representations (1536 if you include both signed zeros)."
This actually had a significant impact on the implementation of IEEE754 on the DEC Alpha chips. The DEC Alpha microprocessors used a software trap whenever a denormal was encountered.
This was a fine idea--until you start emulating an x86 and discover that one of the big programs you want to support--AutoCAD--buries things into denormals and your performance is dog-slow.
Of course ... That's true of any decimal number representation with an exponent. Why is it a problem?
> There are 255 possible representations of zero. They are all considered to be equal.
There are also 255 representations of almost all representable numbers. For example, 10 is 1 x 10^1 or 10 x 10^0 – or any one of 253 other representations. Aside from the fact that you're wasting an entire byte of your representation, this means that you can't check for equality by comparing bits. Take a look at the the assembly implementation of equality checking:
https://github.com/douglascrockford/DEC64/blob/master/dec64....
The "fast path" (which is ten instruction) applies only if the two numbers have the same exponent. The slow path calls subtraction and returns true if the result is zero. The implementation of subtraction falls back on yet another function, which jumps around even more:
https://github.com/douglascrockford/DEC64/blob/master/dec64....
For most comparisons (no, comparing numbers with the same exponent is not the norm) it will take around FIFTY INSTRUCTIONS TO CHECK IF TWO NUMBERS ARE EQUAL OR NOT. Many of these instructions are branches – and inherently unpredictable ones at that, which means that pipeline stalls will be normal. All told, I would expect equality comparison to typically take around 100 cycles. It's not even clear to me that this implementation is correct because at the end of the subtraction, it compares the result to the zero word, which is only one of the 255 possible representations of zero. The lack of a single canonical representation of any number is just as bad for other arithmetic operations and comparisons, if not worse.
Crockfords bugaboo with IEEE 754 floating-point is bizarre, verging on pathological. He devoted a section in his book "JavaScript: The Good Parts" to a rather ill-informed rant against it. When I saw him give a talk, I took the opportunity to ask him what he thought would be a good alternative to using IEEE 754. His answer was – I shit you not – "I don't know". Apparently this proposal is the answer. No thanks, I will stick with the amazingly successful, ubiquitous, thoroughly thought out standard, that was spearheaded by William Kahan – one of the greatest numerical analysts of all time. Anyone who doesn't appreciate how good we have it with IEEE 754 should really read "An Interview with the Old Man of Floating-Point" [1], in which Kahan relates just how messed up this stuff was before the IEEE 754 standardization process. It should also be noted that there already is an IEEE standard for decimal floating-point [2], which is not only infinitely better thought out than this drivel, but also is already implemented in hardware on many systems sold by IBM and others, specifically for financial applications.
[1] http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.ht...
[2] http://en.wikipedia.org/wiki/Decimal64_floating-point_format