News for hackers. Stuff that matters.
Re:My C code works with -O3 but not with -O0 by tyingq on Thursday January 16, 2020 @07:33AM (#22063889)
Does that verifier have a strong possibility of false positives? I'm curious why C compilers have such a strong history of making reasonable checks optional and hidden behind a bunch of switches.
Re:My C code works with -O3 but not with -O0 by hannob on Thursday January 16, 2020 @07:42AM (#22063936)
No, actually the false positive rate of these flags is practically zero. (I'm not sure if it's 100% zero, but I used those extensively, reported many bugs and every time a developer told me "this is a false positive" they were wrong.)

The reason they aren't enabled by default is that that's not what they're designed for. They have a significant performance impact, you can't enable them all at once, they conflict with other security features and they may introduce security issues.

These are developer features. They aren't there to run your production code, they are there to test during development and bug finding.

Re:My C code works with -O3 but not with -O0 by favorited on Wednesday January 22, 2020 @03:12PM (#22121513)
I know TSan has zero false-positives, because it only flags a data race when multiple threads access the same memory without synchronization (and at least one of those accesses is a write). Not sure about the other LLVM sanitizers.
Re:My C code works with -O3 but not with -O0 by saagarjha on Thursday January 16, 2020 @08:10AM (#22064118)
Here's one that I encountered that I cannot figure out:

  $ g++ -x c++ -fsanitize=undefined -
  #include <iostream>
  
  int main() {
      int *a = nullptr;
      std::cout << std::addressof(*a) << std::endl;
  }
  ^D
  $ ./a.out
  <stdin>:5:29: runtime error: reference binding to null pointer of type 'int'
  0
Re:My C code works with -O3 but not with -O0 by Karliss on Thursday January 16, 2020 @08:34AM (#22064334)
You are not allowed to dereference a null pointer no matter how you use it even if you later convert it back to address.
Re:My C code works with -O3 but not with -O0 by hvdijk on Thursday January 16, 2020 @08:44AM (#22064403)
You are allowed to dereference a null pointer, but you are not allowed to access the result or use it as an initialiser for a reference. You can only immediately discard the result, or immediately take the result's address. Using std::addressof(* a) first binds a reference, then uses that reference to take the address, hence the error. You won't get any UBSAN error with &*a.
Re:My C code works with -O3 but not with -O0 by MaulingMonkey on Thursday January 16, 2020 @09:04AM (#22064574)
> You are allowed to dereference a null pointer

Can you cite me something for that? The very first mention of dereferencing in the C++03 standard - ISO/IEC 14882:2003 1.9 Program exection ¶ 4 (page 5) - would seem to disagree:

> Certain other operations are described in this International Standard as undefined (for example, the effect of dereferencing the null pointer). [Note: this International Standard imposes no requirements on the behavior of programs that contain undefined behavior. ]

EDIT: As does 8.3.2 References ¶ 4 (page 136):

> [...] [Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the “object” obtained by dereferencing a null pointer, which causes undefined behavior. As described in 9.6, a reference cannot be bound directly to a bit-field. ]

Re:My C code works with -O3 but not with -O0 by hvdijk on Thursday January 16, 2020 @09:13AM (#22064643)
The standard doesn't say dereferencing a null pointer is invalid. The fact that it gives it as an example of undefined behaviour is a defect in the standard. In the discussion on core language issue #232, the intent has been explicitly stated:

> Notes from the October 2003 meeting:

> ...

> We agreed that the approach in the standard seems okay: p = 0; *p; is not inherently an error. An lvalue-to-rvalue conversion would give it undefined behavior.

http://open-std.org/JTC1/SC22/WG21/docs/cwg_active.html#232

WRT your edit: no, that says dereferencing a null pointer and binding a reference to the result produces undefined behaviour. That agrees with what I was saying. "which" refers to the whole of "to bind it to the "object" obtained by dereferencing a null pointer", not just to "dereferencing a null pointer".

Re:My C code works with -O3 but not with -O0 by MaulingMonkey on Thursday January 16, 2020 @09:20AM (#22064713)
I see! Thanks for the reference.
Re:My C code works with -O3 but not with -O0 by edflsafoiewq on Thursday January 16, 2020 @08:41AM (#22064382)
In C99 however, you are allowed to do &٭a because when the operand of & is the result of the unary ٭ operator

> neither that operator nor the & operator is evaluated and the result is as if both were omitted, except that the constraints on the operators still apply and the result is not an lvalue

Re:My C code works with -O3 but not with -O0 by SAI_Peregrinus on Thursday January 16, 2020 @09:34AM (#22064852)
For anyone who wants it, this is in section 6.5.3.2 Address and indirection operators, paragraph 3.
Re:My C code works with -O3 but not with -O0 by cpdt on Thursday January 16, 2020 @08:39AM (#22064371)
And to elaborate on how the error message relates to this: std::addressof takes a reference which also can’t be null. So when you dereference the pointer you’re actually turning it into a reference (or binding the reference to the pointer, as the error message says).
Re:My C code works with -O3 but not with -O0 by asveikau on Thursday January 16, 2020 @10:55AM (#22065647)
A pretty common way to do the offset_of thing is to take a null pointers. Eg: &foo->member where foo is null.

sizeof on invalid pointer dereferences is also pretty common.

Re:My C code works with -O3 but not with -O0 by lallysingh on Thursday January 16, 2020 @08:32AM (#22064316)
Doesn't addressof() take a reference?
Re:My C code works with -O3 but not with -O0 by zamalek on Thursday January 16, 2020 @01:22PM (#22067299)
> every time a developer told me "this is a false positive" they were wrong

This sounds like a good UB learning experience. The parent comments sounds like the author is running into UB more than they really want to.

Re:My C code works with -O3 but not with -O0 by flohofwoe on Thursday January 16, 2020 @09:11AM (#22064630)
In my experience both clang ASAN and UBSAN (and TSAN, the thread sanitizer) are very solid tools, IFIR I haven't seen a false positive yet (of course some code may be specifically written to rely on undefined behaviour, but of course that's a mine field).

On the other hand, the clang static analyzer may have a shocking amount of messages when run first on a large existing code base, and some of those warnings can be considered more "opinions" than warnings. It's still makes sense and is very rewarding to make a code base "static analyzer clean".

The runtime sanitizers in comparison are very precise and always pointed to actual "sleeper bugs", it's almost definitely a good idea to use them and take their warnings serious.

But anyway, clang ASAN, UBSAN, TSAN and the static analyzer are all really excellent and important tools for everybody writing C or C++ code.

PS: the reason why those checks are optional is that they increase compilation time (sometimes dramatically, like 10x slower compilation or more), and they add runtime instrumentation code which both increases the executables size and decreases performance dramatically (also 2..10x times or more, although the clang sanitizers are really quite fast compared to other solutions).

Re:My C code works with -O3 but not with -O0 by ryandrake on Thursday January 16, 2020 @09:48AM (#22064966)
A lot of developers I've worked with don't want to use these tools because they are working with already-awful legacy codebases that will emit screenfuls of (legit) positives when run, and it feels overwhelming. I remember working on some teams where all compiler warnings were turned off because if you turned on even the default ones, you'd get hundreds of thousands of them during the build, and it looked bad. Since nobody had the budget to spend any time cleaning up warnings because "customers don't care about compiler warnings" they would never get fixed. Forget even sanitizers: You wouldn't even get the splash screen displayed before ASAN would barf on you.

The best projects are the ones that start out from day 1 with all the pedantic warnings turned on, warnings-as-errors, static analysis and sanitizers run as part of the automated build and any peep out of them gets treated just like an error would get treated. When you start the project out that way, there isn't that de-motivating initial hump to get over.

Re:My C code works with -O3 but not with -O0 by rs23296008n1 on Thursday January 16, 2020 @10:28AM (#22065380)
Kind of opposite mindset here.

Current codebase generates around 3000 casting errors. I doubt I'm the only one with this kind of "history" to deal with AND the application is crashing with memory access errors.

What's my cleanup plan for this? Multiple compilers with every warning enabled on multiple platforms. Even a platform we're not targeting. We've mapped out which files and which lines have the biggest code smell. Now we have a giant map on a 65" tv to guide us.

Why all this attention? We're moving this nightmare from 32 to 64 bits. Parts of it were originally 16bit which have already been updated. Cast errors alone are now signposts to other bad code.

Re:My C code works with -O3 but not with -O0 by yjftsjthsd-h on Thursday January 16, 2020 @10:56AM (#22065670)
> Why all this attention? We're moving this nightmare from 32 to 64 bits

It helps that you then almost certainly have buy-in and are allowed to treat this as a priority because it's tied to work that the business is willing to prioritize. You're already over the hurdle of "customers don't pay to fix compiler warnings".

Re:My C code works with -O3 but not with -O0 by rs23296008n1 on Thursday January 16, 2020 @11:34AM (#22066102)
Can't debug anything with a soup of warnings and other mess in the way. They either clear the rot or start from scratch. Our estimate is two years devel to get back to now. So, no time to restart.

Getting rid of the warnings and more importantly the related other debris means being able to use enhancements we make each week. Well... after qa/test have done their thing.

Been good practice for my own indie dev. Treat every warning as an error and make a habit of running a detailed reporting build each day or week. The trick of using other platforms and compilers I brought from years back.

Re:My C code works with -O3 but not with -O0 by pletnes on Thursday January 16, 2020 @09:58AM (#22065051)
Been there done that. One great feature I ran across was that the intel compiler has a #pragma incantation that enables many checks on a per-file basis. This lets you create new code and selectively fix old code without these obvious defects.
Re:My C code works with -O3 but not with -O0 by zwp on Thursday January 16, 2020 @10:55AM (#22065652)
Re:My C code works with -O3 but not with -O0 by wolf550e on Thursday January 16, 2020 @08:07AM (#22064098)
When most people who know C learned C, this technology didn't exist. Currently John Regehr teaches his students that they must use sanitizers.

Note that ruining sanitizers in prod might be insecure. They're for development.

Re:My C code works with -O3 but not with -O0 by wolf550e on Thursday January 16, 2020 @12:04PM (#22066435)
*running, obviously. Too late to edit.
Re:My C code works with -O3 but not with -O0 by jzwinck on Thursday January 16, 2020 @07:37AM (#22063911)
UBSan and ASan have essentially zero false positives. They do point out undefined behavior which happens to work on your platform, but at the very least those are still portability bugs.
Re:My C code works with -O3 but not with -O0 by Liquid_Fire on Thursday January 16, 2020 @07:37AM (#22063915)
This is a runtime check and so has a (small) performance overhead.