News for hackers. Stuff that matters.
Re:Compilers are hard by jcranmer on Thursday January 21, 2021 @12:12PM (#25861270)
The practical effect isn't what you think it is.

Many compiler writers are employed by hardware companies, and these companies are employing them to sell their hardware chips. For these companies, even a 1% boost in performance on $BENCHMARK can drive measurable impact on sales. Even if you're not worried about driving hardware sales, performance tends to be the major benchmark that gets mentioned in comparisons--note how the "Benchmarks Game" (what used to be called the Programming Language Shootout) compares languages solely on the basis of performance.

In other words, if you look at actual customer behavior, performance tends to be one of the most dominant factors in choosing one compiler over another (language compliance is probably more dominant, however). Surety of correctness is not a major factor at all.

Now, in order to actually report benchmarks correctly, you have to implement them correctly, so to some degree you have a basic guarantee that the compiler will usually work. But someone pointing out a soundness issue using a hypothetical example (as opposed to real code) is going to end up low priority because the evidence is that it doesn't matter in practice for the real, large applications that customers care about (as no one else has complained about it for the past several years).

In summary, practical performance will win out over theoretical correctness, but not practical correctness.

Re:Compilers are hard by igouy on Thursday January 21, 2021 @01:49PM (#25862467)
> …"Benchmarks Game" … compares languages solely on the basis of performance.

Not true.

1) source code size is compared

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

2) correctness is checked

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

3) time "to complete and log all make actions" (build the program) is also logged

Re:Compilers are hard by jcranmer on Thursday January 21, 2021 @02:11PM (#25862752)
> 1) source code size is compared

This is new to me (the Benchmarks Game has been around for well over a decade).

> 2) correctness is checked

This is par for the course of benchmarks. A benchmark that is incorrectly compiled provides no useful information whatsoever about performance. It's not really what people mean when they complain about correctness issues in the compiler (which is generally referred to as soundness in literature).

> 3) time "to complete and log all make actions" (build the program) is also logged

Performance in a compiler is usually taken to be a combination of runtime, code size, and compile time, although all three components are not weighted equally.

Re:Compilers are hard by igouy on Thursday January 21, 2021 @02:26PM (#25862996)
Back on May 23 2006, a sortable column of GZip source code size measurements:

https://web.archive.org/web/20060523195137/http://shootout.a...

https://web.archive.org/web/20060519213743/http://shootout.a...

> Performance in a compiler is usually taken to be a combination of runtime, code size, and compile time

The benchmarks game shows source code size, not compiled code size.