No, the bug is thinking that hashing random bytes in your memory is correct. Why wouldn't you make a correct hash function for your struct ?!
The idiom here is to take the address of the struct and read the width of it's whole footprint in memory not just the field data. It's a weak idiom breaking under a common use case.
Most "UB bugs" stem from users who think they know that a struct or data type will be laid out in a certain sequence in memory.
If anything, I'd blame compilers here -- IMO, they should automatically throw at least a warning any time they need to pad/rearrange a struct to make it explicitly clear to developers what's happening.
Even the old language spec from 1989 talks about an "abstract machine with the expressions being evaluated as specified by the semantics". One of the first chapters 1.2 Scope makes this very clear and then 2.1.2.3 Program execution clarifies this even further. Other things to note is that the spec doesn't mention variables being ever stored on the stack, instead it's called automatic storage, see "Storage durations of objects", the word stack is not even mentioned once throughout the whole spec.
Today with modern CPUs all this is even more important, if you think you are operating on a global memory array you are doing it wrong.
Buffer overflows are UBE, too. But the way I proposed the fix is a pretty much cost-free solution, and it's optional.
Redefining C so that struct padding is always 0'd is an expensive solution, and rarely needed.
Using std::vector/array/string::at would literally eliminate buffer overflows and yet programmers aren't generally using this style.
I would love it if I could prove to my colleagues that mandatory bounds-checking would not result in a noticeable performance loss, but my gut feeling is that it's not so. Interestingly Rust (and I guess D) does just that and seems to be getting away with it.
However, in the C UB thread the author of a Rust crate mentioned that a case for using unsafe is exactly this: avoiding the performance loss of bounds checking.
It's still up to you, the programmer. With C, though, you have no choice. No checking for you!
I meant cost-free in the sense that one way or another in C code you wind up passing the length anyway, or in the case of 0 terminated strings, you wind up recomputing it if you don't pass it.
"Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
> Fortunately, the problem of program correctness has turned out to be far less serious than predicted. A recent analysis by Mackenzie has shown that of several thousand deaths so far reliably attributed to dependence on computers, only ten or so can be explained by errors in the software: most of these were due to a couple of instances of incorrect dosage calculations in the treatment of cancer by radiation.
In real life, we use safety gloves, seatbelts, elbow and knee protection, helmets, chainsaw cover, gun lock, ....
Naturally there are those that think such protections are only for children and an accident will never occur to them, until the day they happen to be part of the statistic.
I long for the day that lawsuits for buggy software become a regular activity, only then the industry will actually care to change.
Pure strawman. The statement was that program correctness is far less serious than predicted, not totally benign.
"and that quote in no way talks about bounds checking, which is the issue being discussed here"
Um, programs that exceed array bounds are not correct, so yes, it does talk about them.
[further strawmen not worth addressing]
What's the cost? It's an optional feature.
"Using std::vector/array/string::at would literally eliminate buffer overflows and yet programmers aren't generally using this style."
C programmers don't use that because it doesn't exist in C.
"I would love it if I could prove to my colleagues that mandatory bounds-checking would not result in a noticeable performance loss"
Did you read his article? There's nothing in it about mandatory bounds-checking.
The idea of using bytes is error-prone, but now that you mentioned it, pretty typical of the C mindset.
Of course C++ also has some of these cultural biases. I think they're an important reason why unsafe code continues to be written.
The safe programming tribe, which I include myself, usually refugees from Wirth languages, makes use of C++ abstractions and type safety to deal with unsafety, including type driven development.
Meaning heavy use of templates, type wrappers, STL (or standard library if you prefer) data types, pre-processor only for #include and yes some meta-programming as well.
Then there is the tribe of C refugees, whose C++ compiler was forced on them due to a platform SDK, usually eschew anything standard. Might write some C++ like code due to interop with the SDK APIs and that's it.
Naturally there are a few tribes in-between, but these are the two major groups.
IMO the right solution would be a special annotation on a struct that says “I want the logical value of this struct to uniquely determine the bytes of the struct’s in-memory representation.”
Of course, adding such an attribute without nasty edge cases may be tricky.
Because it's not conformant. Those possible C++ programs are broken.
Conformance is a property of the code, the programmer's decision process is the really interesting thing here to me and I was arguing that such a solution is not unusual for C, but it would be for C++ , where a programmer would avoid it not because it's not conformant, but because it looks odd. pjmpl further clarified that it depends on the type of C++ programmer - I agree.
> I was arguing that such a solution is not unusual for C
You may have asserted that (funny how people call their unsupported assertions "arguments"), but you're wrong if so. Again, it's broken, in C just as much as C++. You wrote
> instead one would use hash functions for each type and in the case of a struct the individual members would be accessed to build the hash value.
But this is just as much true in C as C++, because hashing pad bytes is wrong. In fact, C programmers are probably more aware of this than C++ programmers.
Quote:
Exact-width integer types
The typedef name intN_t designates a signed integer type with width N, no padding bits, and a two's-complement representation.
Thus, int8_t denotes a signed integer type with a width of exactly 8 bits.
The typedef name uintN_t designates an unsigned integer type with width N. Thus, uint24_t denotes an unsigned integer type with a
width of exactly 24 bits.I've stopped programming in C if I can help it. It's a crazy language with support for crazy machines.
But like you said, you have to be aware of this effect at least, and it is not possible to eliminate that by simply clearing all variables to zero. See,
p = (struct ip_weight *)some_used_mem;
p->ip = ip;
p->weight = weight;
At which point should imaginary p->garbage be set to zero? At cast? But that may again do unexpected thing, since casts usually do not modify data. The entire struct abstraction seems leaky as hell, but that’s the price of not dealing with asm directly.These examples show that not C itself is hard, but low-level semantics are. You have to somehow deal with struct {x, y} and the fact that y has to be aligned at the same time. And have different platforms in mind. Maybe it is platforms that should be fixed? Maybe, but these are hardware with other issues that may be even harder to get right.
I think C is okay, really (apart from compilers that push ub to the limit). Type systems in successors try to hide its rough edges, but in the end of the day you end up with semantics of the compiler (c++, rust), that a regular guy has to understand anyway; it’s trading one complexity for another. C++ folks often seen treating it as magic, simply not doing what they’re not sure about. Good part is some languages force you to write correct code, no matter how much knowledge you have. But NewC could e.g. instead force one to create that ‘auto garbage’ to make it clear (why not, safety measures are inconvenient irl too).
I have no strong conclusion, but at least let’s think of all non-cs people who make their 64KB arduinos drive around and blink leds.
The reason for undefined behaviors is to avoid over engineering. In a capable engineer's eye, it is beautiful.
No amount of syntax sugar on top will prevent you from writing unsafe code, unless that basic model is thrown away.
Seriously, just don't cast object of one type to another type and you can forget about alignment until you need to optimize.
You need to keep this in mind basically any time you're going from byte buffers to more complicated types. Basically anything touching allocation or IO needs to be aware of it, IMO, even before optimization.
That C has UBE is not a mistake, it's fundamental to the language design, which allows for unrestricted access to the bare metal. If you want a different sort of language, use Java.
On the other hand, UBE (undefined, or unspecified behavior) are probably the nastiest stuff that can bite you in C.
I have been programming in C for a very, very long time, and I am still getting hit by UBE time to time, because, eh, you tend to forget "this case".
Last time, it took me a while to realize the bug in the following code snippet from a colleague (not the actual code, but the idea is there):
struct ip_weight { in_addr_t ip; uint64_t weight; };
const struct ip_weight ipw1 = {0x7F000001, 1}; const struct ip_weight ipw2 = {0x7F000001, 1};
const uint32_t hash1 = hash_function(&ipw1, sizeof(ipw1)); const uint32_t hash2 = hash_function(&ipw2, sizeof(ipw2));
The bug: hash1 and hash2 are not the same. For those who are fluent in C UBE, this is obvious, and you'll probably smile. But even for veterans, you tend to miss that after a long day of work.
This, my friends, is part of the real mistakes in C: leaving too many UBE. The result is coding in a minefield.
[You probably found the bug, right ? If not: the obvious issue is that 'struct ip_weight' needs padding for the second field. And while all omitted fields are by the standard initialized to 0 when you declare a structure on the stack, padding value is undefined; and gcc typically leave padding with stack dirty content.]