When I went to university in 1982, one of the lower level courses was called "Numerical Methods". It went over all of the issues related to precision, stability, as well as a host of common numerical integration and approximation methods.

I'm just a sample size of one, but isn't this kind of class a requirement for CS majors?

I majored in math and physics. In math, our version of the course was called "numerical analysis," and we covered those things. In physics, the behavior of floating point math was covered in one of our lab courses.

I don't know what's taught to CS majors, and as others have pointed out, programmers don't necessarily study CS.

I believe the issue is just that the pitfalls of floating point are not apparent without a certain level of math education.

But there may be one more pitfall, which is that those of us using FP regularly, also happen to be "scientific" or "exploratory" programmers who haven't learned a lot of formal software engineering discipline (including me). So we understand the math but might be more prone to making mistakes with it.

I do kind of like the idea of flagging any number that is potentially exposed to a FP issue. We all make mistakes. Displaying all floats in exponential notation by default would be a good enough warning to the wary. We only display them as decimals for readability.

It wasn't required at my school a decade or two later. Floating point representations were touched on somewhat in the intro to computer architecture class where we wrote assembly for a MIPS simulator. I got an extra dose because I switched my major to Math, where there was an entire course on Numerical Methods (though it wasn't required by either the Math or CS depts).

I'm a little amazed in retrospect that the Math department was where one had to go to get a class in the mechanical details of computing when as a subject it's usually considered (and often in practice is) notably up the ladder of abstraction. And this was a weird outlier as the single most practical upper division class offered by the Math department at the time...

In my case, the CS class only covered the representation of floating point numbers (a sign bit, exponent bits, fraction bits, issues like bias etc) but not things like numerical approximation or integration methods. Those were in a separate class under the math department. And I think that's fair; after all those are really about scientific computing, not so much about computer science.

> isn't this kind of class a requirement for CS majors?

I think most CS departments dropped numerical analysis from their requirements by the end of 1980s. Nowadays you are more likely to find such a course in some dusty corner of math or engineering departments.

My university moved it from a 100 series course to a 200 series course but it's still being taught to ECE undergrads.

The problem is more that we don't have the tools to track and understand how the errors are altered as we do the math (e.g. how would you even begin to try representing catastrophic cancellation at compile-time) and doing the numerical error analysis on the abstract math itself is hard once the math gets complex let alone trying to figure it out after you've optimized the code for performance & tweaked the algorithms for real-world data/discrete space.

Now perhaps it could be possible to do it at runtime in some way but I suspect the performance of that is prohibitive to the point where arbitrary precision math or decimal numbers is going to be a better solution.

CS isn’t really required for majority of developers.

From a 1998 interview with William Kahan, the “father” of IEEE-754 floating point (emphasis mine):

> My reasoning was based on the requirements of a mass market: A lot of code involving a little floating-point will be written by many people who have never attended my (nor anyone else's) numerical analysis classes. We had to enhance the likelihood that their programs would get correct results. At the same time we had to ensure that people who really are expert in floating-point could write portable software and prove that it worked, since so many of us would have to rely upon it. There were a lot of almost conflicting requirements on the way to a balanced design.

I imagine that the number of people writing code without having taken a numerical methods class has only increased since the late 1970s being talked about, or even in the two decades since that interview.

Not in many of the CS related majors. Though I guess it is for hard CS.

My CS course at a community college went over the limitations of floats in detail, in the CS class.

How detailed, if I may ask?

I remember we learned how the representation worked bit-for-bit, and how it being stored in binary meant it couldn't perfectly represent everything in decimal. 1.01b meaning 12^0 + 02^-1 + 1*2^-2 for example.

There is a proposal about "unum" or "posit" number system. They give more precision for small numbers (small meaning smaller than about 10^70, for 64bit numbers), less precision for huge numbers, and an overall larger range, than the floating point system.

https://en.wikipedia.org/wiki/Unum_(number_format)

http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf

(They are definitely not any easier to understand than the floating point system, though.)

there is also DEC64

http://dec64.com/

> Even without moving away from the IEEE-754 standard, there are ways languages could be designed to minimize surprises.

Floating point in base 10 is already in the standard since 2008:

https://en.m.wikipedia.org/wiki/Decimal_floating_point

The standard is not to blame, the lack of demand for that feature is.

Most of the potential users don’t know that they can demand from their software and hardware suppliers that feature. Using it there would be less “surprises.”

Showing that the result of 9999999999999999.0 – 9999999999999998.0 is a number between 1.9999999999999998 and 2.0000000000000124 will not solve the problem. IEEE floating point doesn't keep track of loss of precision.

To be clear, my point was that if programmers always saw floating-point numbers printed out as a range, from their beginning programming days, more of them would be likely to understand floating-point numbers better — or at least avoid the (impossible) idea that they map 1:1 with the real numbers. Having understood floating-point numbers, they would know what to expect from 9999999999999999.0 – 9999999999999998.0 with 64-bit floating-point. So though seeing a range here won't help magically restore precision that's been lost, having seen ranges earlier would help, before trying to carrying out this calculation.

Nevertheless, you have a good point and what I take away from this is that showing a range for the end result of a computation (instead for a given number directly entered by the user/programmer) can be misleading if the result of the exact computation wouldn't actually have been in that range.

When NASA can't even get it right, because of "surprises", there's no chance in hell I'm blaming us mere mortal programmers... or even 10x wizards. (0)

It's time to look at other ways to depict fractional parts of numbers in a computer. I know that one can express any rational number as a integer fraction. And our computers are incapable of expressing a irrational number exactly - it does so to a certain precision... In other words, every number a computer expresses is a rational number.

The exception is if the computer could express irrational numbers as symbolics, then we could work with the symbolic instead. And then as a last pass, the symbolic could convert to a imprecise rational depiction, or express as its native type.

(0) https://itsfoss.com/a-floating-point-error-that-caused-a-dam...

That failure had nothing to do with floating point specifically. The same failure could have occurred when converting any data type with larger range (including rationals or arbitrary precision numbers) to 16 bit integer.

I would say the fact that floats can have data larger than INT16_MAX is hardly a surprise. That was just a bug, not some great and surprising drawback of floating point.

Rationals get unwieldy quickly, even with the simplest of arithmetic. A couple of additions is enough to get a large denominator.

That has not been my experience using a language with builtin support for rationals. The rational is simplified after each operation so it never grows unwieldy large. It is slower than floats, but imo vastly superior for most use cases.

> The rational is simplified after each operation so it never grows unwieldy large.

We recently had an exponential memory growth bug because rationals can not always be simplified, for example if you start with (2/3) and repeatedly square it. Fortunately, this was not in user-facing code, so there was no chance of a denial-of-service attack, but that's definitely something to watch out for with rationals.

"Most" use cases? You mean the use cases that don't involve heavy math, like graphics, physics, and statistics?

Even simplifying after every operation, in the typical case exact rationals grow exponentially in the number of terms in the computation. This means that either: (a) you cannot use them for any non-trivial computation. (b) you have to round them, in which case they are strictly worse than floating-point numbers because they have redundant representations and a very non-uniform distribution.

Ariane 5 is ESA not NASA

You can work directly in reals.

http://hackage.haskell.org/package/exact-real

That package approximates reals. It fails to do equality on infinite precision reals, for example.