News for hackers. Stuff that matters.
Re:High Performance Computing: Are We Just Getting Wrong Ans... by triska on Sunday September 23, 2018 @04:16PM (#18052931)
In this context, I also highly recommend John Gustafson's book, The End of Error. It is full of great examples, and at one point contrasts advances in numerical computing with improvements in other areas:

"In 1970, a printer might produce something that looks like this, and take about 30 seconds to do so: [picture] Over forty years later, a laser printer still might take 30 seconds to put out a single page, but technology has improved to allow full-color, high-resolution output."

The book makes a compelling argument that working with floating point numbers is similar to printing thousands of low-quality pages per second, instead of getting a single high-quality page in 30 seconds. A few paragraphs later, the book states:

"Eventually it becomes so obvious that change is necessary that a wrenching shift finally takes place. Such a shift is overdue for numerical computing, where we are stuck with antediluvian tools. Once we make the shift, it will become possible to solve problems that have baffled us for decades. The evidence supporting that claim is here in this book, which contains quite a few results that appear to be new."

For more information, see:

https://www.crcpress.com/The-End-of-Error-Unum-Computing/Gus...

and John Gustafson's home page:

http://johngustafson.net/

Re:High Performance Computing: Are We Just Getting Wrong Ans... by nestorD on Sunday September 23, 2018 @04:26PM (#18052982)
He has great examples. Sadly he found shortcomings to his unum numbers and his new alternative[0] do not seem that great compared to the current status quo.

[0] https://posithub.org/index

Re:High Performance Computing: Are We Just Getting Wrong Ans... by jcranmer on Sunday September 23, 2018 @06:23PM (#18053623)
unums are at best very controversial. In actuality, I don't think I've heard a numerical analyst (save for Gustafson) come out in favor of them.

The original proposal that Gustafson had involved a variable-width format, which is going to absolutely kill performance in practice (particularly on the GPU-based supercomputers that are all the rage nowadays). The posit format was essentially a recognition that variable-width doesn't work.

The more serious error is that, well, interval arithmetic turns out not to be useful in practice. Essentially, the potential accumulation of error is exponential in the number of steps in the worst case, so interval arithmetic in practice too often gives you the answer "the result is 1, but it could have been anywhere between -inf and +inf", which is completely useless. He may complain that he doesn't want to use a numerical analyst to get work done, but the rebuttal is that numerical analysis is still necessary anyways, so promising that one isn't necessary is a false promise.

Re:High Performance Computing: Are We Just Getting Wrong Ans... by zlynx on Sunday September 23, 2018 @07:49PM (#18054023)
At least with (-inf, +inf) you know that the answer is nonsense and that you need to go back to the algorithm to find a better order of operations.

With ordinary floating point, all too often people don't realize their results are nonsense. Adding measurement and computation error bounds to their graphs is illuminating.

Re:High Performance Computing: Are We Just Getting Wrong Ans... by triska on Sunday September 23, 2018 @08:07PM (#18054099)
This is indeed a key point in John Gustafson's arguments: With floats, even though there are specialized flags and values that let you detect certain problematic situations, these flags are often not readily accessible in programming languages, and hence these issues tend to go unnoticed.
Re:High Performance Computing: Are We Just Getting Wrong Ans... by jcranmer on Sunday September 23, 2018 @09:30PM (#18054402)
> these flags are often not readily accessible in programming languages

Unless you use C, C++, or Fortran. Which accounts for most numerical algorithms code. Yes, most people aren't aware of them, but not being aware of all the functions in the standard library isn't the same as them not being present there.

Re:High Performance Computing: Are We Just Getting Wrong Ans... by jcranmer on Sunday September 23, 2018 @09:24PM (#18054383)
Just because the error interval is [-inf, +inf] doesn't mean the actual error is [-inf, +inf]. There are other techniques (such as Monte Carlo simulation) that can give much narrower error bounds for the actual computation.
Re:High Performance Computing: Are We Just Getting Wrong Ans... by Something1234 on Sunday September 23, 2018 @11:28PM (#18054821)
How do I calculate measurement and error bounds for my plots when I'm using floating point? I honestly have on idea what to google.
Re:High Performance Computing: Are We Just Getting Wrong Ans... by jedbrown on Sunday September 23, 2018 @05:10PM (#18053234)
>90% of numerical analysis would remain if continuous arithmetic was exact. There exist some prominent failures of finite precision arithmetic (and they resonate with uninformed audiences), but discretization and modeling errors are far more insidious and a deeper challenge for reliable scientific computing.
Re:High Performance Computing: Are We Just Getting Wrong Ans... by angry_octet on Monday September 24, 2018 @04:25AM (#18055923)
Agreed. Though I'd say that model errors are a challenge, but I wouldn't call them insidious. There is generally serious analysis of the effect of the chosen discretization method/grid size, how to handle sub-grid scale effects, etc. Likewise models.

I think it is interesting to compare this old panic with ML work. Without gazillions of flops the neural network approach was dead. After a few years it became clear that float/double precision was useless, and now for certain problems half precision is standard.

Where one resource (people, CPU, ram, money) is the fixed constraint, other factors will be tweaked to compensate.

Re:High Performance Computing: Are We Just Getting Wrong Ans... by jedbrown on Wednesday September 26, 2018 @03:34PM (#18079464)
I don't know what domain you work in. The modest 1986 editorial statement from the Journal of Fluids Engineering is still pretty much a gold standard for rigor in real-world applications and is exceedingly rarely achieved in domains where direct observational data is hard to come by (most of geophysics, much of biophysics, etc.). E.g., https://jedbrown.org/files/20160912-NumericalAccuracy.pdf