Indeed modern GPUs still use fixed-point rasterizers.
Not true in Lua >= 5.3. 64-bit signed integers are included in the number type now. And there is a math.ult function for treating them as unsigned.
And so 0x7fffffffffffffff is representable properly.
Correct. That’s using the wrong base. Decimal floating point doesn’t have that problem.
The general rule is that, in base b, you have finite positional ("decimal") representations of numbers p/q where q is a divisor of bˆn for some n. So e.g. 1/10 doesn't have a finite binary representation because 10 = 2 * 5, and that factor 5 ensures 10 is never a divisor of a power of 2. Likewise, 3 is coprime with 10, so you can't represent 1/3 in decimal.
It _is_ imprecision. Even if you cannot represent 0.3 exactly, you can consider the closest representable number, and that is different than the sum of the closest numbers to 0.1 and 0.2 because doing all of this introduced imprecision.
Lua 5.3 introduced a proper integer type. However, JavaScript still represents all numbers as doubles!
Wellll... Yes and no. And I want to nitpick "FP ops are imprecise" because it's important sometimes.
It does come up in Lua - Lua uses 64-bit double floats for everything, _even array indexing_, because they have 53 bits of mantissa and they're guaranteed to represent all 32-bit integers with 100% precision.
I just opened Lua and got `2 ^ 32 == 4294967296.0` and `2 ^ 32 + 1 == 4294967297.0`. You can store 4 billion things in a Lua table and access them with double float indexes.
The usual "0.1 + 0.2 != 0.3" is _not_ imprecision. You could dedicate 1,000 bits to a float and still find some example of a number that's trivial to represent in decimal but repeats forever in binary.
While I'm on it, fixed-point and floating-point aren't magically different. Floats work better on very big values. Fixed-points are more predictable but run out of range easily when squaring numbers. This comes in 3D math when finding the length of vectors or normalizing vectors.
The PlayStation 1 didn't have jiggly vertices and warpy textures because of fixed-point. It had jiggly vertices because its GPU didn't have subpixel-precise rendering, and it only had 16 bits of precision. It had warpy textures because it had affine texture mapping. The N64's GPU had more bits of precision and it had perspective-correct texture mapping. There's no 16-bit floating-point format that would have saved the PS1 from wobbling.
Also floats aren't the cause of T-junctions or "sparklies" that are common in 3D game levels. That will happen in any system with finite precision, and if fixed-point would fix it, we'd use fixed-point.