The most beautiful FP bug I remember was a denial of service in some webservers where by setting the header "Accepted Language: en-gb;q=0.3 en;q=0.8 ..." to a specific value you could send the official Java floating-point parser in an infinite loop (and this affected several Java webservers).
So at each webpage request, you were sending one CPU core of the webserver basically busy-looping. Hence crashing the webserver in a few requests at most.
Now it'd be heresy if I were to say that maybe, just maybe, had the standards mandated to use 0 to 100 for weighting instead of 0 to 1.0 we could have dodged a whole source of potential issues!?
No, no, I realize this is heresy: let's all keep using numbers that cannot even be represented correctly (except as strings), parse those strings into something approximately representing what's written in the string, then make more approximation errors while doing computation with these numbers, propagation errors, epsilon estimation errors, and keep insisting we all could have authored "What every computer scientist should known about floating-point numbers" (which is 80 pages long and, well, approaching a treaty) ; )
/rant off
I think a protocol that used integers instead of decimal point would have led to more correct implementations. So it would be a better protocol.
If the protocol had nothing to do with it, I don't think you'd see the same bug come up in as many implementations
I absolutely blame the protocol. Fixed point isn't widely available and used, but integers are. Someone below says protocol spec goes to three digits past the decimal. Using a constrained integer would have allowed the same range of values, but without the complexity of decimal numbers; that's a protocol mistake.
In addition to the ease of using floating point numbers to incorrectly represent decimals, there's also fun with formatting, comma vs period, number of digits to report, etc.
Maybe using a float is overkill in this case, since you're never going to hit more than a hundred languages. But it's at least not setting any limitations.
qvalue = ( "0" [ "." 0*3DIGIT ] )
| ( "1" [ "." 0*3("0") ] )
There are 1001 valid values with 1117 representations. Only in practice, Firefox clamps to 2 decimal places, and it used to clamp to 1 decimal place (https://bugzilla.mozilla.org/show_bug.cgi?id=672448), and who knows what other software does.
And it gets worse: the grandparent suggests there are servers that parse it as floating point. Do they accept q=1e2? If they accept it, are there clients that send it? Do you need to be compatible with those clients?
I also would invite you to find a similar instance where a parsing bug for integers crashes the server. Not throws an exception, but falls over in an uncatchable way. I'm not sure I have ever seen one.
NaNs/infs are particularly vicious, as they propagate through computations.
-- IEEE754/64-bits: Interfaces.IEEE_Float_64
-- And here we strip out non-numeric values.
Use Interfaces;
Subtype Numeric_Real is IEEE_Float_64 range IEEE_Float_64'Range;
https://www.youtube.com/watch?v=VLTqIXZ1jQQ#t=15m00s
I never had a PS1 but it always bugged me when playing on someone elses.
Conversely I like the fact a lot that it does. And some others that probably have fond memories of PS1 too recreated this on modern computers using shaders.
Early PS1 games fit the entire world in to a 24 bit space, and you can tell that there is steping if you drive very slow in a game like Ridge Racer. Later games moved the world around in the coordinate system to manage precision.
Another reason why the graphics looks extra "popy" on PS1 (And early DirectX games) is that vertices where computed at a pixel accuracy. OpenGL, Doom, Quake and more modern hardware have sub pixel accuracy.
We made a simple 3d demo for uni using fixed math (and we were clueless then and weren't using macros because sometimes you could refactor some operations out that way and we wanted all the speed). There were lots of bugs caused by this.
Can't imagine how painful writing a whole game this way would be.
Basically you use the platform's native types (e.g. uint32) and decide which of the bits are for the integer part and which are for the fractional. uint32 can be interpreted as 20/12 for example. Your register went from representing "units" to representing "2^-12 increments of an unit". The ALU can do sum/comparison/subtractions transparently, but multiplications and (god forbid) divisions require shifting to return to the original representation.
The choice of how to split the bits is a tradeoff between range and precision. It can vary from one routine to another. It's a pain to compose code with it.
Short story: native float types are a blessing for programmer productivity and library interoperability.
1111.1111 is (1 + 2 + 4 + 8) + (1/2 + 1/4 + 1/8 + 1/16)
I'm fascinated by how common it is for programmers to hate floats. Yes, if you write business software for a living you may not have much use for them. But there's a lot more to computing than business software.
Hate floats, love big decimals
Hate OOP, love functional
Hate RDBMS, love nosql
Hate html web pages, love SPA
In my opinion it's a confluence of influences.
1) the same social media attitudes poisoning society generally. "I've got an opinion, and it's worth just as much as yours". News flash - opinions are like arseholes, everyone has one.
2) Inexperience - devs who have only ever built chat apps and don't understand why relationships hence RDBMS are common and useful in line of business applications. Devs who have not tried to capture the water level in a tank or the voltage from a solar panel and don't get why floats are useful.
3) resume padding. nuff said
Although to go down that road, schemaless databases preceded RDBMS by decades, and then were replaced for good reasons.
Not necessarily true. Theoretically you can replace floats by fixed-point representations given enough bits, and in practice this also works often.
It's a pity languages/hardware don't have good built-in support for fixed-point numbers though.
What did you gain from this, exactly?
I'm sure there are some special situations where fixed point is better, but for general purpose scientific and technical computing, floating point is obviously the right choice.
On modern CPUs, fixed point is slower and error prone (both in usage and the computations themselves) relative to floating point.
I don't think it's a pity that fixed point support is out the door. It sucks. Floats minus subnormals are the least surprising numerical representation out there, and the most versatile.
Do you really want to worry about whether your numbers are saturating or wrapping around? Keeping track of what your max/min are? Tracking decimal places? Implementing division by hand? Screw all that. Fixed point is awful and is essentially an academic exercise today.
Similarly, fixed-width integers have a totally different algebra than true integers, and yet they're immensely useful.
Floating-point numbers behave like a physicist would do calculations rounding at every step, but doing everything in binary. Or hexadecimal if you prefer, it's equivalent, but still difficult to get your head around to. Then there additionally is + and - 0 and + and - infinity, and several settings for how rounding is done exactly.
This is incorrect. To count as an approximation, there have to be some accuracy bounds. This is impossible to define, as the reals don't have the same cardinality as any floating point number system.
Now, for many interesting and useful real-valued functions, a float-valued function can be defined that is an approximation. But there is no general way to map a function defined on the reals to a function defined on the floats with a known accuracy.
There's a bunch of work on discretization of models after you've done optimizing them - that works, you can get high-performance inference with low-bit fixed point numbers; but that's after the learning has been done using proper floating point (however, you don't necessarily need high accuracy floating point, e.g. single precision may work betted than double precision simply because it's twice less bytes to copy over).
That's the reason why they introduced https://en.wikipedia.org/wiki/Bfloat16_floating-point_format
It's similar to normal half precision float but the exponent is the same as with 32bit float and mantissa has been sacrificed to get room for it.
From Microsoft BASIC to Javascript, many programmers have worked in languages that only have floats. Financial calculations involve exponential/log math much like scientific problems. There are issues with rounding and representation there (e.g. there is no such thing as $0.01 in floating point, only $0.50, $0.25, $0.125 and some sum of 1/2 fractions that comes very close to $0.01 and even reads in and writes out as $0.01 even though it isn't.)
Don't forget control systems.
If you only use those (maybe including division, I'm not sure), and don't rely on hardware support for sqrt, sin/cos/tan, approximate-inverse, etc., you can totally use them deterministically between different architectures.
And computing integers IS faster than computing floats, at least on a CPU.
In the abstract yes. In reality not so much. A lot of what floats are used for in games is vector math (you are often in a 2D/3D world), and vector math in fixed point requires a lot more work in integer math since you constantly need to shift down things in order to avoid overflow. Overflow bugs are a constant problem when doing dot products in fixed point integer math.
Another common operation in vector math is square root (to normalize vectors) Modern CPU's do that very fast, and you can also use clever approximations that use the way floating point's are represented in hardware.
Most of whats done in 32 bit floats in games, needs to be done with 64bit integers to manage these problems if you decide to use integers, and that means that your data grows, more cache misses, and things slow down.
On top of this everything you do in a game needs to be drawn on the GPU so you need to convert everything to floats anyway to show them on screen, and converting between integers and floats is also slow.
If you try to do with Fixed Point Arithmetic what was intended to be done with Floating Point, you're the problem.
A 32 bits integer can hold values up to 4 billions, if I have that kind of value in a simple game, then yes i will switch to Floating Point Arithmetic, but when does that use case happen if you're not writing a physic simulation game ?
All the time. Let me give you an example from a game I made. During early R&D I used 32 bit numbers and a fixed point of 11 bits. That means that a normalized vector is between 1024 and -1023. You can do 2 dot products between vectors before it breaks it breaks. (10 + 10 + 10 bits plus one sign bit). That means that you have to do a lot of shifting down. the world can only be 20 bits large, because you need to be able to multiply world coordinates with vectors without getting overflow.
20 bis is very low resolution for a real-time world. You get problems with things not being able to move slow enough at high frame rates. (this was in 2D). Switching to 64 bit was the right move.
> some game engines
and
> simple games
A 3D game is not a simple game.
> the approximation is generally good enough for a game
No, it's not generally good enough for a game – only for some games, and not good enough for any game with 3D graphics.
Using fixed point with 11 bits after the decimal, with numbers as low as 100 (18 bits) we already have errors as large as 100·(1/100)=0.977.
This goes so far, that Intel added an AVX512 instruction to expose this 52/53 bit multiplier for integer operations, to accelerate bigint cryptography calculations like RSA and ECC.
A separate, complicating factor is that floating point math often don't yield the same results with different levels of optimization turned on. The reason to use integer math for games is because you want it to be deterministic so can be an issue.
> All conforming implementations of this standard shall provide the operations listed in this clause for all supported arithmetic formats, except as stated below. Each of the computational operations that return a numeric result specified by this standard shall be performed as if it first produced an intermediate result correct to infinite precision and with unbounded range, and then rounded that intermediate result, if necessary, to fit in the destination’s format (see 4 and 7).
I have a suspicion that on a fundamental level, floating point isn't actually good for games or machine learning. They're just used because existing computers are so good at floating point number crunching, especially GPUs.