News for hackers. Stuff that matters.
Re:The Strict Aliasing Situation Is Pretty Bad by eternalban on Tuesday March 15, 2016 @02:04PM (#11291688)
This was an eye opener for me. Bell Labs tech is as usual a mountain of hidden complexity hiding under a "simple" facade.

No more excuses for me. Time to learn Rust.

Re:The Strict Aliasing Situation Is Pretty Bad by aidenn0 on Tuesday March 15, 2016 @05:10PM (#11293018)
C is actually quite simple, even the aliasing rules (I think the aliasing rules all fit on about a quarter page). Programming in C though is anything but.

The tension between weakly typed pointers and the desire to generate efficient code is where there is a problem. More or less anywhere you violate the aliasing rules you are doing something that Fortran doesn't allow at all, and by disallowing it semantically the hope was that C programmers could have their cake and eat it too. The reality is that all systems code should probably be compiled with alias analysis disabled.

Re:The Strict Aliasing Situation Is Pretty Bad by xorblurb on Tuesday March 15, 2016 @06:13PM (#11293423)
I completely agree.

A spec of a quarter page can already be incredibly convoluted and with very hard to anticipate consequences -- even more so when crazy people are interpreting it without caring about the consequence of their acts in the real world. And C is not just about aliasing rules; the current situation is that ANY undefined behavior is a landmine waiting to kill you, regardless of whether is seems to makes sens for your target architecture. And that is mostly because compiler writer have an insane interpretation of the standard: the definition of "undefined behavior" is "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"

A key word here is "nonportable" and it is clearly not considered often enough by some compiler writers, who generally prefer to see all undefined behaviors as a licence to "optimize" your code without carrying too much about warning you about potential bad side effects because it's "hard" according to them.

This does not make even the beginning of any sense. If it is not what many programmers are expecting (a majority ? -- most coworkers I know including my direct boss are not even aware of all that mess), is costly during the translation phase, has unclear/unquantified runtime performance benefits, is dangerous in the real world and is hard to detect when bugs are activated by those "optimizations", then WHY they are doing them in the first place? From an engineering point of view this is just plain insane. Correct executions and in depth safety are extremely valuable, and only becoming more so year after year, and when they pretend that it's not their fault that programs are breaking they are being even more ridiculous; a compiler does not exist in a vacuum, and neither just to reproduce itself and satisfy the curiosity of geeks for mathematical logic.

Obviously some amount of alias analysis can be useful, and this is clearly one of the topic really intended from scratch in the standard to address some performance issues, but maybe it would be enough to explicitly identify what you want to not alias. Seeing the bug when they discuss about allowing uint8_t to not behave in general as an unsigned char in regard with aliasing is just plain disgusting and makes me lose yet again a portion of the tiny remaining trust I had in them.

It will soon get to the point that C/C++ will not be realistic languages to consider if you want any kind of reliability. Maybe I'm even deluded in thinking this is not already the case.

Re:The Strict Aliasing Situation Is Pretty Bad by aidenn0 on Tuesday March 15, 2016 @08:38PM (#11294145)
I get a bit fed up when software developers complain about C compilers and blame the developers, as though compiler developers are somehow a different breed or something. It's all software, and compilers are actually one of the easier things to write.

Somehow, the developers of gcc, clang, and various commercial compilers are all crazy, while people who work on any other project in C are sane? Why haven't the sane software developers forked an open-source compiler and implemented sane semantics?

Blaming the state of compilers on compiler developers without understanding their motivations for making engineering choices is intellectually lazy, particularly for other software developers.

Re:The Strict Aliasing Situation Is Pretty Bad by xorblurb on Tuesday March 15, 2016 @09:43PM (#11294418)
What you write is all very meta, so maybe enlighten us on their motivations and why they should continue in that way? And on how using a language is the same as implementing it and how it seems that the application space has magically folded into a single point last night?

I know already too much for my taste about the reasons they gave to come with such horribly risky designs. Example: "it's difficult to properly warn where we are doing dangerous optims". I'm not buying it. Don't do them in the first place or use more heuristics. Publish your data about your risk / benefit analysis. I think they made strategic mistakes. "Everybody" in the academia and security / safety related business is saying that. And no, that won't be solved with dynamic checkers. They should concentrate on optimizations minimizing the risk of new behaviors concretely appearing, and heuristics maximizing the detection of the intent of the programmer. That sounds informal as hell, because this is, but in the real world solving a problem does not always mean finding the perfect solution to an equation, and good enough approx are often the best thing to seek. Compiler writers know that. They are just not seeking the good thing. The "smartness" they think they are creating though their imaginative use of undefined behavior is as smart as my ass, because on real projects those kind of sufficiently smart compilers are indistinguishable from an adversary.

I have recently implemented some kind of compiler, though not C to machine code. In the real world it's not easy. It's messy. It's made of blood and tears. You have to care about all kind of little details for your users. You have to make the junction between two domains that sometimes have quite different semantics -- there is never a perfect solution to that. And all of this, with the minimal risk to be misused. If you don't care for your users, their is no point. I've the greatest respect for the authors of CLang, which I used for the front end. It's not perfect, but it is quite good and it does the work. But sometimes, when you feel that somebody effort really is misplaced, you better tell him (or the community, my voice is not particularly original), and explain your reasoning. Otherwise, you effectively won't be entitled to complain latter, if you wait too long.

So now what do we have: two domains, C and various target architectures, that used to be with quite some low impedance mismatch, by design, are considered radically different. The expectations of most users have nothing to do anymore with the expectation of how the compiler writers thinks the users should write their code (or have written it in the past, for less maintained software). That won't ends well. Actually, we are already in the mess, while we don't especially needed more than we had already.

So go on and please explain your POV. But anyway, no, I won't fork and write a "friendly C" compiler overnight. Neither will maintainers of cryptography softwares, probably. I'm just yet another datapoint that will loose some time in adding all the -fdisable-insanity of the day anywhere he passes. Because somebody else somewhere else in the world thought that would be a good idea to infer proofs using rules written for the least common denominator of all computers to detect if you read the n1570 over and over again enough times to obtain the privilege to get a sane translation of your code that actually will anyway only ever run on x64_86, thank you very much.

The same debacle somehow happened about memory models (see how they have been received for kernel work). If you only ever care about the abstract, concrete real prospective users won't greet you as you might have expected. Rightly so. Especially when your formalism has been proven unimplementable and unsound.

Re:The Strict Aliasing Situation Is Pretty Bad by the_why_of_y on Wednesday March 16, 2016 @08:18AM (#11296674)
I don't think it's going to be a fruitful exercise to demand that implementers of ISO C/C++ change their implementation to guarantee certain properties of programs that ISO C/C++ consider to be invalid, and thereby weaken their competitive position vs. other implementations.

The root cause is the ISO C standard itself, so if you want any change in that direction, the best approach would be to join the ISO C working group and make proposals to replace various undesirable undefined behaviors with, at the minimum, implementation-defined behaviors (starting with signed integer overflow, perhaps). Probably there aren't any vendors of one's complement machines and the like left to veto your proposals.

The C11/C++11 memory models for the most part don't solve a problem that kernel developers have, because all kernels already contain tested solutions to the same problem, so one would not expect kernels to quickly exchange all their tested and highly performant concurrency primitives. The only benefit for them is that the memory model effectively prohibits implementations from doing some optimizations that would be valid only under the assumption that there is only one thread of execution, and this benefit is implicit, i.e. you don't need to use any of the new language features or standard library headers to benefit.

Re:The Strict Aliasing Situation Is Pretty Bad by aidenn0 on Tuesday March 15, 2016 @10:52PM (#11294651)
First of all, I think that many people underestimate the performance advantages of some of these "optimizations" I've seen real-world loops that take a 40%+ reduction in speed by disabling alias analysis. All of those fancy loop optimizations that everybody agrees are good fall apart if you can't prove that various objects in them don't alias.

Secondly the way people shop for compilers is completely broken. It's often nearly entirely done by running the candidate compilers against a suite of benchmarks and picking the one with the best numbers. Usually only a single -O option is passed. Disabling any of these "optimizations" by default means you lose.

Thirdly I think you are overestimating the number of people that care about this problem. John Regehr ran a bunch of crypto code through clang's undefined behavior detection tools and reported bugs. Many of those bugs were closed with a WONTFIX because the code generated today is correct!

Lastly, the ISO spec is what it is, and has what it has. To the extent that compilers are in competition with each other, a single vendor working on a separate dialect of C is spending effort that helps it not at all in the benchmark game, and which over 90% of their users will never use. GCC being the 800lb. gorilla it is, occasionally users of other compilers request language extensions from GCC, but most often their checkboxes are "Complies with the ISO standard" and "generates good code" I would love for the next version to have a more friendly C dialect as an annex, and then everyone who cares could just pass --std=c17friendly and it would work across all compilers.

[edit]

As far as my original post being very meta, that's mainly because none of what I'm saying above has anything to do with my primary complaint: people who aren't writing compilers are complaining about decisions made by those who are, and attributing it to something fundamentally wrong with those people. Compiler developers aren't insane or stupid. Since most C compilers are self-hosting, compiler developers use their product more than most other developers. Saying they have tunnel-vision is perhaps a bit more close to the truth, but it is worth examining the environment they are in, as it's not like there is a "will be a compiler developer" gene that selects for compiler writing skill along with "doesn't give a shit about users"

And most of this isn't directly pointed at you, but rather the fact that there's been an unusual amount of compiler-writer hate on HN recently. I thought I could let it go by, but I hit my limit.

[edit2]

To sum up my first edit; all of my "meta" post I feel like I could have said even if I didn't know a lot of compiler developers. I don't really know any web developers well, but I don't assume that the 3MB of JS that is loaded for tracking and advertisement on most large sites is because web developers are insane or stupid. I assume rather that at least some of them are trying to deliver the best product within the constraints of their customers (a page that can be monetized, and completed in under a certain number of billable hours).

Re:The Strict Aliasing Situation Is Pretty Bad by quotemstr on Friday March 18, 2016 @02:59PM (#11314024)
Ahaha. Rust is language that assumes dynamic memory allocation never fails. The way current compilers treat undefined behavior is terrible, but Rust is worse.
Re:The Strict Aliasing Situation Is Pretty Bad by steveklabnik on Friday March 18, 2016 @03:41PM (#11314339)
The Rust language knows nothing about the heap.

The standard library does assume this, but it's not an inherent language issue. Most people who are in resource constrained environments won't be using the standard library anyway.

Re:The Strict Aliasing Situation Is Pretty Bad by quotemstr on Sunday March 20, 2016 @07:51PM (#11325289)
Rust also made the same mistake that Go did: it tried to appease the "exceptions are bad" crowd and ended up adding exceptions as an afterthought. I'm angry: Rust is merely good, but with a few early tweaks, it could have been great.
Re:The Strict Aliasing Situation Is Pretty Bad by kibwen on Sunday March 20, 2016 @08:09PM (#11325343)
Rust does not have exceptions, not even in the limited capacity that Go sort-of has. It's legal for a Rust implementation to translate panics into aborts, which means that they cannot be relied upon as an error-handling mechanism. Furthermore, the means to halt unwinding in Rust exists only to prevent memory unsafety from occurring when Rust is embedded in another language via the C interface, because unwinding across an FFI boundary is undefined behavior. In fact the exact mechanism used in the aforementioned role has deliberately arbitrary restrictions placed upon it in order to prevent anyone from using it as a general-purpose error-handling mechanism. Finally, the prevailing Rust culture overwhelmingly discourages using anything other than Result for error handling, to such an extreme that I've never in all my days even seen anyone attempt to "catch" a panic as an error-recovery mechanism.
Re:The Strict Aliasing Situation Is Pretty Bad by quotemstr on Sunday March 20, 2016 @08:19PM (#11325388)
https://doc.rust-lang.org/beta/std/panic/fn.recover.html looks a lot like catch to me. That the documentation suggests not using it as "catch" is merely a political statement. While it may be legal to turn panics into abort, I doubt implementations will do that, because by doing so, you'll break programs that assume that panic recovery works.

The reason Rust's standard library isn't safe against malloc failure is that propagating Error everywhere would be cumbersome; I distinctly recall rust mailing list discussions on this point.

The only reason handling allocation failure has to be difficult is that the Rust people put themselves into giving into the anti-exception people. Once you allow exceptions, allocation failure becomes as easy to address as any other kind of resource exhaustion.

The Rust designers made a serious error eschewing the only fully general and fully ergonomic error handling strategy we've found.

Re:The Strict Aliasing Situation Is Pretty Bad by kibwen on Monday March 21, 2016 @12:18AM (#11326212)

  > the Rust people put themselves into giving into the 
  > anti-exception people
It's a shame that you characterize design decisions as "giving into" the crowd instead of trying to understand why the decision might make sense in the context of Rust. Designing the error handling story was a discussion that spanned years, with dozens of discussions, many implementations, and likely hundreds of participants.

  > That the documentation suggests not using it as "catch" 
  > is merely a political statement
It's not just political, note the "RecoverSafe" bound on the type parameter. See https://doc.rust-lang.org/nightly/std/panic/trait.RecoverSaf...
Re:The Strict Aliasing Situation Is Pretty Bad by dbaupp on Sunday March 20, 2016 @08:46PM (#11325494)
Implementations will do that, e.g. https://github.com/rust-lang/rfcs/pull/1513 .
Re:The Strict Aliasing Situation Is Pretty Bad by quotemstr on Sunday March 20, 2016 @09:08PM (#11325563)
Encouraging this style of programming makes me sad.
Re:The Strict Aliasing Situation Is Pretty Bad by kibwen on Monday March 21, 2016 @12:33AM (#11326260)
This isn't a style of programming, it's a compiler flag for end-users. There's a reason that GCC and Clang have the -fno-exceptions flag.
Re:The Strict Aliasing Situation Is Pretty Bad by quotemstr on Monday March 21, 2016 @01:21AM (#11326389)
The -fno-exceptions flag also makes me sad, because it encourages people to write libraries that are worse than they could be purely for the sake of interacting with code that chooses not to use an important feature of the language. Exception support should not be optional.
Re:The Strict Aliasing Situation Is Pretty Bad by jpco on Monday March 21, 2016 @08:22PM (#11333440)
What's wrong with Rust's error-handling mechanism (namely, the `Result` type)? Generally when writing Rust, panics only come up when you actively invite them with something like an `.unwrap()` call or the `try!` macro, which is Decidedly Bad Style for anything serious.
Re:The Strict Aliasing Situation Is Pretty Bad by kibwen on Monday March 21, 2016 @11:33PM (#11334251)
The `try!` macro never causes panics, it uses the `Result` type as is typical for error handling in Rust.