The fact that C arrays decay to pointers without any bounds is single-handedly responsible for a huge chunk, possibly even the majority, of all RCEs, worms, malware, and exploits. Ever. In the history of computing.
It was a bad design.
It was a bad design in 1978.
It was known to be a bad design in 1978.
Other languages knew that checking array bounds was important, including for security. The internet made the impact of using C much more devastating but people were exploiting buffer overflows in the 80s to great effect. Some of C's predecessors/contemporaries passed a length as the first part of an array so bounds-checking was possible, though that has the downside of not being able to pass slices of an array without copying.
C could have included an arrayref type that was a length + base pointer, and let array l-values decay to an arrayref instead of a pointer. Then taking a slice of an array would not require copying elements. You could still take the address of an individual element. This would not have required much work to implement, even in 1978! Maybe the first compilers didn't insert array bounds checks, but at least the entire design wouldn't preclude them. Let's say you even spell arrayref as []. It would mean sizeof() works on arrays passed to functions.
void wat(int[] values) { for(int i = 0; i < sizeof(values); i++) { printf("look ma, no buffer overflows! %d", values[i]); } }
(Yes, I know this is not K&R syntax)
Maybe you can forgive C for the stupid header compilation model (why let the compiler do what you can make the programmer do by hand?). You can understand why they might not have foreseen the need for namespaces. D&R didn't invent the macro system so that's not even their fault.
What is unforgivable is the horribly stupid design of C's arrays.
I actually think it would be beneficial if the standards committee added arrayref now. It won't fix all the busted C code but at least you could start improving the #1 problem. Compilers could eventually adopt a flag to prohibit arrays from decaying directly to pointers. You'd probably have to introduce lengthof() to avoid confusion and use some other syntax to declare one, maybe array(int) or something.
When C was designed, and even today, there are systems without pipelining, where it is expensive (in time) to de-reference a memory address and follow that pointer.
I do not argue that the design you suggest would be safer, and even have advantages for slicing; but that's really not the kind of program that C was intended to service writing.
Also, C is supposed to scale down to //really// simple systems. Systems that lack indirect addressing modes, caches, MMUs, etc. It is literally intended to be a thin veneer over actual assembly for those systems, and why so many operations are specified in terms of /minimum standard unit size/ (for portability of that almost machine code between systems).
What you advocate is more like what C++ actually /should/ have been; a reason to use something more than C to gain advances in safety and ease of design.
It's questionable whether people wanted that performance though, at least when it resulted in less security. About bounds checking in ALGOL 60: https://en.wikipedia.org/wiki/Bounds_checking
A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interest of efficiency on production runs. Unanimously, they urged us not to—they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous.
There's no question about it, the "ANSI C Rationale" makes it very clear what they considered "the spirit of C"[1]:
> - Trust the programmer.
> - Don't prevent the programmer from doing what needs to be done.
> - Keep the language small and simple.
> - Provide only one way to do an operation.
> - Make it fast, even if it is not guaranteed to be portable.
> The last proverb needs a little explanation. The potential for efficient code generation is one of the most important strengths of C. To help ensure that no code explosion occurs for what appears to be a very simple operation, many operations are defined to be how the target machine's hardware does it rather than by a general abstract rule. An example of this willingness to live with what the machine does can be seen in the rules that govern the widening of char objects for use in expressions: whether the values of char objects widen to signed or unsigned quantities typically depends on which byte operation is more efficient on the target machine.
> One of the goals of the Committee was to avoid interfering with the ability of translators to generate compact, efficient code. In several cases the Committee has introduced features to improve the possible efficiency of the generated code; for instance, floating point operations may be performed in single-precision if both operands are float rather than double.
[1] http://www.lysator.liu.se/c/rat/title.html Quoted section is found here: http://www.lysator.liu.se/c/rat/a.html#1
-- http://www.memorymanagement.org/mmref/lang.html
Adding runtime bounds checking of automatic storage arrays (i.e. arrays on the stack) is relatively easy in C, at least until the compiler runs into illegal type punning. The real problem in implementing these compiler safeguards comes with crossing translation units, or with heap blocks. There's a reason languages like Rust and Go rely heavily on static linking and stack allocation; it's more difficult or more costly to implement those safeguards when the compiler can't see all the source code, or pointers pass through an opaque layer. Nothing in C precludes automatic bounds checking of all array access, via fat pointers or lookup tables. Fabrice Bellard's Tiny C compiler implemented precise bounds checking for both automatic and dynamic storage-allocated objects a decade before UBSan and ASan. Even deriving an invalid pointer crashed the app at the precise point where it happened. That widely-used C compilers don't do that is a strong hint there are other, real-world constraints in place.
Also, in language like Java it's not uncommon to see people reinventing dynamic heap allocation using char arrays, susceptible to all the same overflow problems. When you see people doing that, that should be a hint that a language like C might work well.
I don't understand all the C hate. Then again, I have no problem employing various languages according to the task, or creating DSLs. I suppose if I was wedded to a single language or to the idea of a single language, C would look much worse to me.
This is untrue: Rust certainly does not do any optimisations linking statically by default, nor is there a difference between putting an array on the stack or on the heap. While it is true that code can benefit from whole-program optimisation, it isn't the default in either language, just like it isn't the default in C.
LTO notwithstanding, once you add those more sophisticated constructs, iterating the language becomes more difficult. You don't hit upon the best method for implementing various types the first time, or the second time, or even the third time. glibc is backwards compatible for programs compiled over 15 years ago (GCC's fixinclude hacks notwithstanding). You'll never see that with Rust's or Go's standard library, just like you never saw that with C++.
My point wasn't that static linking was necessary. My point was that static linking is indicative of other tradeoffs that most people don't understand. Static linking isn't just about making packaging easier. It's also about making it easier to write and implement the compiler and standard environment.
My more abstract point is that people who think C is on its last legs don't understand the whole picture. There's nothing intrinsic to C that makes it unsafe. Febrice's compiler was perfectly capable of implementing the C standard to the letter. What makes C unsafe are the requirements found in the niches where C exists, and those requirements don't magically disappear because the name of the language changes.
Rust supports unsafe code, but implementing code in Rust which is rigorously robust in the face of OOM situations, or where you need to implement use-case memory management strategies requires relying almost exclusively on unsafe code. (Try using Rust without boxing, for example, as is necessary if you want to catch OOM.) If you don't need those things, you probably don't need a low-level language, either. I love C, but I also love language like Lua with lexical closures and stackless coroutines. To me, languages like Rust and even C++ exist at a middle ground that is very unappealing to me.
C isn't standing still, either. Strategies like SafeStack (see http://dslab.epfl.ch/proj/cpi/) can provide substantially the same safety guarantees as Rust in terms of real-world attack vectors, without having to modify any existing C software, and without giving up performance.
None of this is to say languages like Rust are useless. Just that the harms and inevitable demise of C per se are, IMHO, greatly exaggerated. And if and when a language like Rust grows in usage, I doubt it will supplant C so much as open and populate virgin territory.
That paper indicates that you do in fact give up performance, and the performance is comparable to existing SFI techniques. SafeStack itself is insufficient to prevent UAF problems with the heap. CPI prevents them, but with significant overhead. And you still don't get full memory safety.
It's not necessary, you can plug in a custom allocator that works differently and use boxing as usual.
There are plans for more robust custom allocator APIs that make this even easier to handle.
Also, really, even if Rust didn't have this, the situation wouldn't be worse than C. In C you have to malloc and free things manually. In Rust you can do that too. Rust's abort-on-OOM is an stdlib thing (which can be overridden as previously mentioned).
The performance hit is generally negligible, especially with abstractions like iterators in Rust that avoid them entirely, and standard optimisations that can lift the checks out of loops... optimisations that do not need any of the things you say that the compilers want. The cost of calling code in a different dynamic library (e.g. getting the dynamic symbol address and then doing the actual call) is going to be much greater than whatever bounds checks it does in almost all situations.
> There's a reason languages like Rust and Go rely heavily on static linking and stack allocation.
As I just said, this is factually false. Static linking is entirely orthogonal to bounds-checking optimisations (neither Rust nor Go do whole program optimisations when linking statically, so it can't be the motivation for it), as is putting data on the stack. GC seems even more irrelevant, especially to Rust which doesn't have one.
> My point was that static linking is indicative of other tradeoffs that most people don't understand. Static linking isn't just about making packaging easier.
But it isn't indicative! In Rust's case, linking statically is for packaging: the reason is the ABI is unstable, so dynamically linking is very annoying to manage and many of its benefits are inhibited.
> There's nothing intrinsic to C that makes it unsafe.
The forever-growing list of CVEs caused by basic mistakes in C code says otherwise. Things like overrunning a buffer or reusing a freed pointer are not at all caused by domain specific constraints, they're the price one pays for using 40 year old technology. You can see this in modern tools that try to assist with getting safer C: they are often using things that didn't exist when C was created. (And, don't get me wrong, C is here to stay, even if all new C development was stopped today, and so efforts to make it safer are very good, but at some point we have to face the reality of C/stop the C-apologism.)
> Febrice's compiler was perfectly capable of implementing the C standard to the letter.
This is essentially meaningless for two connected reasons: the major problem with C is the holes in the standard (undefined behaviour)---not compiler bugs---and, people want fast code, they need optimisations, which often exploit undefined behaviour.
> (Try using Rust without boxing, for example, as is necessary if you want to catch OOM.)
Boxing or not is irrelevant to safety: using Box allows in fact more aggressive `unsafe` code (one can rely on address-stability to correctly sidestep the compiler's normal checks). Rust-the-language knows effectively knows nothing about the stack or heap when reasoning about safety: it does reason about stack scopes, but it doesn't care where the data is actually positioned in memory: Box<T> is isomorphic to a plain T in this respect.
In any case, the power of Rust is the ability to wrap code into safe abstractions: if there is a particular feature the standard library doesn't provide (yet), external libraries have the power to create APIs that have the same level of safety, maybe with a bit of `unsafe` internally. You can see this even in "use-case memory management" situations like a kernel: http://os.phil-opp.com/modifying-page-tables.html
> That widely-used C compilers don't do that is a strong hint there are other, real-world constraints in place.
Yes. Those constraints are self-inflicted wounds caused by the fact that C wasn't designed for this. If you have a proper iterator API, a culture of unsigned array indexing, widespread use of a size_t equivalent instead of int for loops, etc. etc. these issues vanish.
This is C snake oil, sold by the C community that usually omits the fact that computers build a decade before the PDP-11 like the Burroughs, already had much better systems programming languages like ESPOL and NEWP, two Algol derivatives.
Algol 68 already had slices and modules (Algol68 RS) among many other nice features, running in early 70's hardware.
There are a few other examples.
This model enables binary-only distribution of libraries, you get the code as a .a (or lib, .so, .dll or whatever) and the API declaration as a header file.
You can write code against a library without having the library, using only the header. You can't do the final linking of course, but you can write the code.
The alternative, I guess, would be to embed this information in the library itself, and have the compiler extract it, which sounds as if it would have been scary from a performance point of view 40 years ago (and also somewhat hard).
I used to think it was hopeless, especially as each new language that came out required garbage collection or worse targeted the JVM. Perhaps cloud services will motivate people to fix this stuff since now computing costs are a hard line item on the books.
We surely did know that Burroughs was selling an operating system written in ESPOL, later NEWP in 1961. Nowadays Unisys still sells them as MCP.
We did know that the Flex machine was written in ALGOL 68RS in 1980.
We did know that VME was written in S3 in 1970.
We did know that Pilot was written in Mesa in 1977.
We did know that Lillith was written in Modula-2 in 1997.
There are lots of other examples.
The main difference was that UNIX and consequently C, source code were available for free because AT&T could not sell it, while people had to pay for the other ones or they were behind research walls.
There's still a lot of C code out there, and a lot of new C code still being written.
I don't agree: it's been a long process, but the trend is unmistakable. It's hard to remember now, but in the early '90s C and C++ were completely dominant. Nowadays they're much more specialized: you're as likely to build your company on Java or even Python/Ruby as you are to build it on C++. People talk about how it's hard to hire C++ engineers nowadays, while in the '90s "C++ engineer" was pretty much synonymous with "programmer". And so on.
The interesting question is what will mobile devices, the IoT and embedded devices in general be programmed in? C and C++ are popular choices today, so the trend is not really "unmistakable".
Everything else usually has other languages available as well, one just needs to search for what is out there.
Some of today's dominant platforms are developed mostly in Java (see Android), and web development targets the LAMP stack. This means that the business is centered in ventures that exclude most languages, not because there is technical merit on other alternatives.
I'm sure it's possible to gather some individuals that are more than willing to badmouth Java and Python with a passion with the same ease we see here people complaining about C.
Oberon for ARM Cortex-M4, Cortex-M3 Microcontrollers and Xilinx FPGA Systems
http://www.astrobe.com/default.htm
Pascal and Basic for lots of micro and pico-processors
http://www.mikroe.com/compilers/
Ada,
http://www.ghs.com/products/ada_optimizing_compilers.html
http://www.ptc.com/developer-tools/apexada
Java for MCUs
A lot of the things rust brings to the table aren't always relevant on embedded platforms. Dynamic memory allocation on embedded is the exception, not the rule. Everything is statically allocated, so memory management is relatively simple -- everything sticks around forever.
The things that make C/C++ good for embedded are sorta what make it unfortunate for general purpose use. The things that make Rust/Go/Swift good for general purpose use make it unfortunate for embedded use.
In that case, you can simply not use the dynamic allocation features of Rust, just as you can simply not use malloc() in C.
> GC languages
> Dynamic memory allocation on embedded is the exception
Right, I was referring to Go and Swift with the comment about GC. Since rust didn't have that problem, I mentioned why major adoption might not be forthcoming.
I understand that you can statically allocate all you want in Rust, but it's memory safety, particularly with regards to object ownership is one of its major selling points. Object ownership and lifetimes are trivial when everything is static.
The other arguable selling point to rust is its standard library, but much like C++ the standard library would be left aside in most embedded applications.
So it doesn't buy you much of anything at all, but it takes some work to setup, plus you are fighting the momentum that C/C++ has. Unless there is some other compelling reason to use it, I don't expect much adoption.
Rust is a better choice, and it's designed with this in mind. It's still young, though, and I think there might be some as-yet-unsolved issues (these are things I've vaguely heard of and could be totally off-base) like binary size, ease of dealing with raw pointers, etc.
If I was doing this sort of programming for a personal project, I'd probably try using Rust, because I like it.
Dunno about Swift, though IIRC the current reference implementation may also currently rely on GC.
> like binary size, ease of dealing with raw pointers, etc
As for raw pointers, they're exactly as capable as raw pointers in C, though they're deliberately more verbose as well, because even in embedded contexts one should be favoring references over raw pointers, since references are still fully checked for safety even in embedded mode and yet are represented by raw pointers at runtime and hence have zero runtime overhead.
Rust is very promising language, but unless you plan to write everything from scratch, you have to depend on 3rd party driver implementations for most stuff. Databases for example. There isn't a single database vendor that has Rust drivers.
BTW I am not implying that is Rust's fault, actually I can't think a syntax that would make it less verbose, and I am a huge Rust fan.
I wasn't aware of the tools that do this automatically. Just found one and it looks promising.
something like C::function_from_c() is one option
Rust is unsuitable as a replacement for C because its memory management is poorly thought out (ie. its a joke). Here's the relevant paragraphs from the Rust FAQ. Really?
"Rust avoids the need for GC through its system of ownership and borrowing, but that same system helps with a host of other problems, including resource management in general and concurrency.
For when single ownership does not suffice, Rust programs rely on the standard reference-counting smart pointer type, Rc, and its thread-safe counterpart, Arc, instead of GC.
We are however investigating optional garbage collection as a future extension. The goal is to enable smooth integration with garbage-collected runtimes, such as those offered by the Spidermonkey and V8 JavaScript engines. Finally, some people have investigated implementing pure Rust garbage collectors without compiler support."
This is a library thing, not a language thing.
If single ownership is enough for you, go ahead and use it. But if you need a different memory management strategy, that is available too.
Rust, the language, provides a single clear memory management strategy. It also provides the ability to design your own abstractions for different strategies, and implements some of these in the stdlib.
C/C++ have refcounting and GC libraries too. Does that make them a joke?
> They really don't know what direction to go.
Does the presence of libraries for refcounting or even GC in C (most famously Boehm) mean that C has an incoherent story around memory?
> Rust's multiple methods of memory management makes it strange, at best,
> for programmers to decide how to build a program or an API.
> Are the owners of Rust planning on adding a GC? Really?
The second is something like Bohem: if a system wants to use GC for some reason, they have an interface to add a GC'd type. But Rust proper, the language, will not have GC. It's completely contradictory to the goals of the language.
To address this specifically: Current plans for Rust are mostly along the lines of adding the bare necessities in the stdlib to allow GC implementations to be written.
There are mostly-niche use-cases for having a GC in Rust. I've written some of the motivation here[1] (note that that blog post is about a pure library GC independent of Rust, which is different from what is planned; but the motivations are similar).
One major use case is if you want to talk to a language which has a GC. Say you're writing a native extension to a Ruby or Node and want to deal with the GCd types within Rust code in a safe way without pausing the GC. Or if you're writing an interpreter for a GCd language. Or you're writing some code that deals with complicated cyclic graph-like datastructures.
These are all pretty niche, but the workarounds in these cases aren't pretty so it's nice to have some form of GC capabilities in Rust. This is not a price you pay by default, and it's not something that affects anyone but the people who need these types. It will probably take the form of some low level APIs that use LLVM stack rooting to collect roots, and some traits in the stdlib, which can be used by an independent GC library (not part of the stdlib) or a language bindings library (also not part of the stdlib).
Rust itself will never get a GC as part of the language.
[1]: http://manishearth.github.io/blog/2015/09/01/designing-a-gc-...
We also have nearly 40 years of infrastructure built on C, which needs to be maintained and updated.
This is the same old argument advocating for rewriting everything from scratch just because someone somewhere managed to develop a new flavor of the month.
There are plenty of reasons why the whole world still has a heavy demand for COBOL and FORTRAN developers, and the development of new flavor of the month isn't a good enough reason to eliminate this demand.
I'm not saying rewrite everything for no reason. I'm saying that there are reasons, and we've gotten a very good idea of what those are over the last 40 years.
But C++ won't be easy to replace, and I'm not sure it needs to be, since rewrites are highly risky, time consuming and disruptive. With some luck and depending on how the language evolves we might be moving from C++ to a safer C++.
But using the C++ features that make it safer than C is only an option in small security motivated teams.
Sadly the majority of C++ teams, at least in the enterprise space, tends to use it as "C with classes" thus voiding most improvements the language has to offer over plain C.
C++ is split between multiple factions. I'm doubtful that the one programming in C with classes is interested in learning e.g Rust.
I do agree that we should be moving away from C and C++, though. It's pretty simple, really: C was a pretty good language in 1978. We didn't know a lot of things in 1978 that we do now in 2016. It now makes sense to revisit those decisions in light of nearly 40 years of practice. The so-called "PL Renaissance" has given us a whole host of new languages which have steadily chipped away at the dominance of C and C++, and I think this is a healthy trend that ought to continue.