News for hackers. Stuff that matters.
Re:“Fiercely resist any further broadening of the scope of t... by pcwalton on Thursday April 14, 2016 @12:59PM (#11498397)
> You can't just shrug this away with "Well, if you want performance...", because people in fact don't want abstract "performance"... they want the language they were truly writing in to perform well, not for what is de facto a different dialect of the language to suddenly appear and replace the language they were using.

Is wanting a for loop setting an array to zero to optimize into memset optimizing the language they were truly writing in? I think it is. But that optimization frequently depends on undefined behavior.

UB exploitation usually exists because people filed bugs on compilers complaining that they didn't optimize some case they expected to optimize.

Re:“Fiercely resist any further broadening of the scope of t... by nkurz on Thursday April 14, 2016 @01:46PM (#11498883)
Is wanting a for loop setting an array to zero to optimize into memset optimizing the language they were truly writing in? I think it is. But that optimization frequently depends on undefined behavior.

Yes, this is a good optimization, since it efficiently does what the programmer intended. The bad optimization is removing the security-essential loop altogether when the compiler notices that result appears unused, and sensitive information is left susceptible to later attack.

UB exploitation usually exists because people filed bugs on compilers complaining that they didn't optimize some case they expected to optimize.

I doubt that anyone has ever filed a bug saying "I explicitly wrote a loop to zero memory, but the compiler failed to optimize it out." If you know of one, please point to it. I think you are throwing out the baby (intentional C) with the bathwater (autogenerated C++).

Re:“Fiercely resist any further broadening of the scope of t... by pcwalton on Thursday April 14, 2016 @01:50PM (#11498931)
> I doubt that anyone has ever filed a bug saying "I explicitly wrote a loop to zero memory, but the compiler failed to optimize it out."

That optimization is a natural consequence of SROA and DCE. If you claim you don't want those optimizations, I don't know what to tell you. Those optimizations are some of the most basic, critical optimizations any modern C/C++ compiler does and throwing them away can easily result in at least a 2x performance loss.

Re:“Fiercely resist any further broadening of the scope of t... by cwzwarich on Thursday April 14, 2016 @03:37PM (#11499713)
The optimization actually has very little to do with SROA. In LLVM there is a special loop idiom recognition pass designed to detect memset / memcpy loops that works after ordinary SSA conversion.
Re:“Fiercely resist any further broadening of the scope of t... by kbenson on Thursday April 14, 2016 @02:05PM (#11499047)
> I doubt that anyone has ever filed a bug saying "I explicitly wrote a loop to zero memory, but the compiler failed to optimize it out."

I suspect it's less that they asked for that than they asked for some generic optimization on loops, and the implementation works great 99% of the time, and 100% of the test cases, but because it was approached from a performance perspective the idea that sometimes we want to change the state of the memory, even though it has no affect on a correctly functioning program, isn't considered. Zeroing memory in this way has nothing to do with correctly running programs, the only use is to defend against your program malfunctioning and exposing the data, or some external actor looking at memory. Neither of those have to do with the normal functionality of the code the compiler is running, but is is important. Unfortunately since it deals with things that have nothing to do with the actual instructions needed to make the program function, it's easy to overlook.

Re:“Fiercely resist any further broadening of the scope of t... by pcwalton on Thursday April 14, 2016 @02:09PM (#11499092)
> because it was approached from a performance perspective the idea that sometimes we want to change the state of the memory, even though it has no affect on a correctly functioning program, isn't considered.

A lot of dead stores happen because of post-inlined, post-constant-propagated (etc.) code. Nobody writes that code by hand; it occurs due to inlining and other aggressive types of optimizations. You need those optimizations for performance; IPO is a huge win.

It's like the old joke: "Who writes that kind of code? Macros do."

Re:“Fiercely resist any further broadening of the scope of t... by kbenson on Thursday April 14, 2016 @02:30PM (#11499249)
I understand, I'm not really taking a stance. The question, really, is should the compiler be responsible for making sure your unused bits and ends of memory that are provably irrelevant to the functioning of the program that is being compiled stay exactly as you left them? I think it's interesting because the situation is a bit untenable on both ends of the spectrum.

If we disallow all those optimizations, we end up with hugely wasteful programs in the current C ecosystem. Enforcing this is useless, the market will route around your best intentions, because no company that's making money will be willing to cede a huge performance gap to their competitors.

If we require code be annotated to bypass optimizations, we run afoul of the fact that it's impossible to know what optimizations might be developed in the future that will affect portions of our code that might be security sensitive.

The problem is insidious and deep. Imagine an architecture that allows specialized instructions to speed up zeroing chunks of memory, but does so by hardware mapping different pages around behind the scenes (to swap in a pre-zeroed page for a filled page). Correctly detecting that the architecture has an efficient method for zeroing large chunks of space could be detected and used by the compiler, but the hardware is working against us because the original memory may still be accessible to an external actor. C is ensuring the memory addresses you requested are zeroed, it's just not ensuring that the data that existed within them is no longer anywhere on the system.

Re:“Fiercely resist any further broadening of the scope of t... by Kristine1975 on Friday April 15, 2016 @01:41AM (#11502439)
The C standard already allows the compiler to remove calls to memset if the memory isn't used afterwards by the program (essentially dead-store elimination). C11 provides memset_s to clear memory for security reasons.
Re:“Fiercely resist any further broadening of the scope of t... by cwzwarich on Thursday April 14, 2016 @03:08PM (#11499507)
You can easily write a for loop that optimizes to memset without relying on undefined behavior. The only potential problem is that the loop bound evaluation could require a memory access, but this is simple to work around.
Re:“Fiercely resist any further broadening of the scope of t... by pcwalton on Thursday April 14, 2016 @03:17PM (#11499567)
Well, not if the compiler can't prove that the stores don't alias the induction variable. TBAA is the easiest and most reliable way to prove that, which is UB exploitation.
Re:“Fiercely resist any further broadening of the scope of t... by cwzwarich on Thursday April 14, 2016 @03:33PM (#11499680)
No, that is not the most reliable way to prove that property. The most reliable way is to ensure that the loop bound and induction variable are local variables that have not had their address taken.
Re:“Fiercely resist any further broadening of the scope of t... by pcwalton on Thursday April 14, 2016 @03:46PM (#11499774)
Well, yes, in the trivial case, but not if the induction variable and/or loop bound come in as references (as in templated code, etc).

There's also the issue of the store changing the array base, as in Chris Lattner's example on "What Every C Programmer Should Know".

Re:“Fiercely resist any further broadening of the scope of t... by cwzwarich on Thursday April 14, 2016 @05:30PM (#11500493)
You can always make the induction variable and loop bound local variables, and the induction variable is almost always a local variable anyways. It was the WebKit coding style for years to use local variables for loop bounds for this reason.

This is the example you're talking about in "What Every C Programmer Should Know":

  float *P;
  void zero_array() {
    int i;
    for (i = 0; i < 10000; ++i)
      P[i] = 0.0f;
  }
If you make the global variable static, then it gets converted to a memset with -fno-strict-aliasing, because the 'P' variable never has its address taken. If you used LTO with sufficient linkage information, the same thing would happen. Similarly, if you pass the array as an argument then the loop gets converted to a bzero call just fine without -fno-strict-aliasing:

  float *P;
  void zero_array_impl(float* a) {
    int i;
    for (i = 0; i < 10000; ++i)
      a[i] = 0.0f;
  }
  void zero_array() {
    zero_array_impl(P);
  }
I don't think this is a very good example for these reasons.