News for hackers. Stuff that matters.
Re:What every compiler writer should know about programmers ... by DannyBee on Thursday March 03, 2016 @06:26PM (#11220862)
One of the things this person misses is a lot of these are undefined with the explicit goal to let compiler authors take advantage of them.

So his real argument is with the C committee for not defining these things the way he wants them. His claim that the behavior of what should happen is obvious is pretty strange (IE "it should do exactly whatever the compiler is trying to prove it doesn't")

It's essentially a complaint that compiler authors should follow a mystical spirit of C that he seems to have in his head.

(Plus, you know, he's kind of a dick about it all)

I read things like "We had plans to turn this optimization into a production feature, but eventually realized that the GCC maintainers only care about certain benchmarks and treat production programmers as adversaries that deserve getting their code broken. ... The current GCC and Clang maintainers suggest that we should convert programs to “C”. How would that turn out for Gforth?"

I was intrigued, so i went looking for the mailing list discussion where the gcc or clang people suggested this or where any of this happened.

I can't find it. On either set of mailing lists. I searched for gforth, i searched for anton, etc Nothing.

Re:What every compiler writer should know about programmers ... by kazinator on Thursday March 03, 2016 @08:34PM (#11221432)
> lot of these are undefined with the explicit goal to let compiler authors take advantage of them.

Constructs should be undefined because the current landscape of C implementations is surveyed, and those constructs are found to be nonportable to such an extent that their behavior cannot be even "implementation-defined" or "unspecified".

That's the beginning and end of it, period.

That's why certain thing were undefined or unspecified in the first ANSI C standard. For instance, different compilers had different evaluation order for function arguments, without documenting which, or guaranteeing that it would always be the way it is.

The committee didn't sit there and decide "let's make evaluation orders undefined for the sake of blazing speed, even though all C compilers today are doing it left to right. Let's have it so that after 1989, they can use whatever order they want!"

It was, "Ooops, we can't codify a specific evaluation order, because looking at what's out there, there isn't one. If we were to codify an order, implementations not having the order that we codify would have to be modified to conform with our standardized language."

As time goes on, these stupid things should be tightened. Over the course of thirty years, it's okay to codify some gratuitous lack of definedness in a language area and make everyone straighten it out in their implementations.

Re:What every compiler writer should know about programmers ... by loup-vaillant on Friday March 04, 2016 @04:30AM (#11222816)
I believe the order of evaluation of function arguments is unspecified (though it might lead to undefined behaviour in some cases, I'm not sure).

---

Signed integer overflow is more interesting. In most platforms, it is very well defined: it either wraps around or it saturates. On one's complement machines I don't know, though I'd expect a deterministic (yet mostly nonsensical) result.

So it might sound reasonable to have signed integer overflow be implementation defined. Except that another reasonable behaviour is to trigger a trap, just like divide by zero. So they put in into the "anything can happen" bucket, that is, undefined. And now we have compilers taking advantage of this, by assuming signed overflow simply never happens. Which sometimes lets them deduce interesting things such as `k<k+1` evaluates to `true` no matter what.

I'm not sure the standard bodies anticipated that. I bet they were thinking something along the lines of "the runtime can assume it never happens, and fail to perform the relevant checks in the name of performance". I'm not sure they originally interpreted this as "the compiler can assume it never happens, and treat the alternative as dead code".

Overall, I see 3 kinds of undefined behaviour: the most innocuous one is when the runtime is allowed to trigger a trap, and crash the program right away. The second one is when the runtime is allowed to not perform the relevant check, and go bananas because of mangled memory or something. This one is more serious, and the source of many security vulnerabilities. The third one is when the compiler can assume the relevant undefined behaviour doesn't even happen, and make lots of interesting deductions from this assumption. This one is mostly insane, though I can understand why it would be nice in some limited cases.

We need 3 buckets for undefined behaviour. "Anything can happen" is too coarse grained.

Re:What every compiler writer should know about programmers ... by nkurz on Thursday March 03, 2016 @06:50PM (#11220980)
While his tone may be unwarranted, I think his claims are more reasonable than you suggest. I'm not sure where the actual bug reports are, but here's a long but excellent thread on comp.arch where Anton makes his point more fully: http://compgroups.net/comp.arch/if-it-were-easy/2993157

Edit: In another comment, you say "The whole thing is like complaining that people follow the law, instead of the spirit of the law." I think that might be accurate. Impractical as it may be, there are many (including me and presumably Anton) who believe that society benefits if individuals hold themselves to a higher standard than obeying the letter of the law and flaunting the spirit.

Re:What every compiler writer should know about programmers ... by DannyBee on Thursday March 03, 2016 @06:58PM (#11221014)
Whether his claims are warranted or not, his vitriol is still directed at the wrong place. If the behavior he claims is so easy to define is so easy, obvious, and beneficial, as he claims, he should just submit defect reports and get it fixed. Blaming the compiler authors is just silly (in the sense that it will never change anything, and is not effective)

As for folks "holding themselves to a higher standard", it's not really a sane thing to do here, because there is no universally agreed upon, or heck, even "precisely defined"set of "bad things" that they should not do. Depending who you ask, you will find differing answers on what people think should not be taken advantage of.

IE despite his claims, it's not even clear what this "higher standard" is supposed to look like.

Re:What every compiler writer should know about programmers ... by marssaxman on Thursday March 03, 2016 @07:09PM (#11221046)
His point as I understand it is that the people who would be doing the fixing are unlikely to agree that the defects he is reporting are actually defects, and that reporting them is therefore futile, absent a shift in consensus on the topic of undefined behavior. The paper exists in order to help promote that shift.
Re:What every compiler writer should know about programmers ... by DannyBee on Thursday March 03, 2016 @07:13PM (#11221064)
This is not an effective way, at all, to promote such a shift, of course. It's also not like this is the first paper. There are plenty of papers (he even cites a bunch) saying the same thing. So i'm not sure why anyone would think this would do much ...

Convince any of the 100+ vendors producing gcc forks to do it, have their customers sing their praises, and others will follow.

yelling about it on the internet is not likely to do much :)

Re:What every compiler writer should know about programmers ... by MaulingMonkey on Thursday March 03, 2016 @10:35PM (#11221912)
> One of the things this person misses is a lot of these are undefined with the explicit goal to let compiler authors take advantage of them.

It doesn't look like it's missing it to me:

"C compiler maintainers also claim that these “optimizations” give speedups for other programs. However, that would require programmers to convert their C* programs to “C” first, a process which can produce worse code (Section 3), and more importantly, requires an effort that would be much more effective if directed at source-level optimizations."

The goal is to let compiler authors take advantage of undefined behavior for, among other things, performance. The counterclaim is that these goals are noble, but that letting compiler authors take advantage of undefined behavior as a means to achieve those goals is, unfortunately, counterproductive.

> Convince any of the 100+ vendors producing gcc forks to do it, have their customers sing their praises, and others will follow.

So... yell (positive things) on the internet?

> yelling about it on the internet is not likely to do much :)

Oh. Well, for what it's worth, I agree. As such, I spend my energy investing in staying the hell away from C when possible. I then yell positive things on the internet about alternatives for my own amusement. Still doesn't do much, but it does a little.

Re:What every compiler writer should know about programmers ... by abecedarius on Thursday March 03, 2016 @10:14PM (#11221848)
My reaction was to stop coding in C. You're safe from immediate irrelevance, but the alternatives keep getting better.
Re:What every compiler writer should know about programmers ... by forthy on Friday March 04, 2016 @02:46PM (#11226391)
Anton is a compiler writer himself (part of the Gforth team, which I lead). We do submit bug reports. They get rejected as "invalid", on UB. It is pretty clear that the way we use GCC to implement Gforth is C* and not "C", we use extensions to GCC as source level optimizations. Without them, Gforth would be a factor 3-5 slower than it is now. We experience slowdowns in that order when we have to switch back to "C" instead of finding a workaround for that particular "optimization".

So sane behavior of the compiler is absolutely necessary to create fast programs. The tone reflects more than 10 years of frustration about "C" maintainers, and their inability do deal with the needs of their customers, and our long-term goal is to completely throw out C, writing our own code generator instead. It's going to give us another factor 2-5 (depending on the problem; some microbenchmarks might gain a factor 10) over the C*-based solution we have now. And, at the same time, we can forget about releasing a bug workaround patch for Gforth for every new GCC release, as it was the case for GCC 3.x and 4.x (GCC 5.x is better, some of the critics has been heard). This is feasible, as the number of popular architectures has decreased so far that a small team can support the remaining ones (which is essentially x64, ARM, and ARM64, with some variations within the family).

As a GNU project, we understand ourselves as customers of the other, very important GNU project, GCC, of which we, and almost everybody else in the GNU project, depend. If it's broken, it is too bad. And if the bug report process is broken due to the "UB=we can blow up your program" attitude, it's worse. If the competing comiler maintainers, clang, have the same attitude, we can't even switch.

We are also part of a language standard effort (Forth, of course), so we know how such a committee think; this is not different just because the language is different. What we don't understand is why the C standard has a lot of UB, and only little IB (implementation defined behavior). For example integer overflow: This is a very obvious candidate for IB, as the behavior of the underlying hardware defines integer overflow; usually as wrapv in two's complement, sometimes as trapv, but the other options like one's complement and such are non-existent. The result of such an operation is never "undefined", and as wrapv two's complement is the dominating implementation, it is pretty portable, as well.

Or how pointers look like (pointer comparison): You know how your pointers will look like when you implement a C compiler, so you can fully specify the behavior of pointer comparisons. It's IB, because segmented architectures like 8086 with their overlapping segments simply don't give reliable and fast results. But if you don't program for MS-DOS, you don't need to care: usually pointers are just register-sized integers pointing to a flat, non-segmented memory, and pointer comparison is just compiled to integer comparison.

Bottom line: If you don't understand what your underlying machine does, and how your code generator translates statements to instructions, and therefore you think that x-1>x never can become true, you shouldn't be writing compilers.

Re:What every compiler writer should know about programmers ... by 95014_refugee on Thursday March 03, 2016 @10:42PM (#11221940)
"So his real argument is with the C committee for not defining these things the way he wants them."

... where you conveniently leave out the part where the 'C committee' is stuffed with compiler vendors.

"Plus, you know, he's kind of a dick about it all"

If you had been on the receiving end of what these people call 'optimisations' for more than a few years, you'd think he was being pretty civilised.

Re:What every compiler writer should know about programmers ... by rbehrends on Thursday March 03, 2016 @08:31PM (#11221422)
> So his real argument is with the C committee for not defining these things the way he wants them.

To some extent yes. But if you take it to that extreme – where you really have to understand the standard at the language lawyer level –, C becomes an unreasonably complicated language where you can only trust experts to have a deep enough understanding to write correct code.

[deleted]
Re:What every compiler writer should know about programmers ... by DannyBee on Thursday March 03, 2016 @07:27PM (#11221113)
FWIW: With nothing meant to denigrate what happened here, in general, if you want real answers, use the mailing list ;)

The people hanging out on IRC are random llvm developers, random llvm users, and some random others.

Re:What every compiler writer should know about programmers ... by erichocean on Thursday March 03, 2016 @07:31PM (#11221136)
I've deleted my post, but in this case, I was speaking to two actual LLVM developers, not just random dudes on IRC. Anyway, it's not their fault, "get what you pay for" and all that, and like I said, I like and use LLVM and really can't complain. :)
Re:What every compiler writer should know about programmers ... by conceit on Thursday March 03, 2016 @07:17PM (#11221082)
> (Plus, you know, he's kind of a dick about it all)

It's called tongue in cheek. Like so: you dick :P

Re:What every compiler writer should know about programmers ... by marvy on Thursday March 03, 2016 @07:43PM (#11221188)
I don't think he's being tongue in cheek. I bet you he's genuinely angry and resentful. From his point of view, he once had a lovely language, C*, and he can't use it any more, and is being forced into assembly instead, for no reason other than the perversity of the maintainers of the only two C compilers that he feels that he can use.
Re:What every compiler writer should know about programmers ... by conceit on Thursday March 03, 2016 @07:47PM (#11221213)
> I don't think he's being tongue in cheek. I bet you he's genuinely angry and resentful.

Oh you are right, that's two totally contradicting modes that combined are not at all sarcastic. Not at all.

Re:What every compiler writer should know about programmers ... by conceit on Thursday March 03, 2016 @10:26PM (#11221884)
The title even is a play on the referenced http://blog.llvm.org/2011/05/what-every-c-programmer-should-... how is that not tongue in cheek? Kindly go vote yourself down with a rake.
Re:What every compiler writer should know about programmers ... by marvy on Friday March 04, 2016 @06:58PM (#11227675)
I don't have enough karma to get a down vote button. Even if I did, I'm not sure that voting on your own posts is allowed. I up voted you instead.

Yes, the title is a play on the llvm post. I could be wrong about this, but the way I took it was like this:

"No, don't you presume to tell me what I need to know about undefined behavior. I am your customer, and here's what you need to know about me. Stop treating me like an enemy."

But you could be right. I'm bad enough at detecting jokes and sarcasm in person with people I know. Over a textual medium with a stranger, I don't really trust myself much.