But I'm sure, Real Soon Now(TM), if teams just adopt (best practices / better libraries / better code reviews / better static analysis tools / better testing) all those problems will be solved and C will be a great language to keep using.
After all, saving a few extra CPU cycles is totally worth breaking all SSL encryption and leaking private keys, quite literally putting the security of EVERY SINGLE FUCKING HUMAN WHO HAS EVER TOUCHED THE INTERNET in jeopardy.
So sure, let's just keep using C. It's fine, why change anything amirite?
(Sarcasm aside: In the general case it is not possible for humans to write security-critical code in C. So far the history of the internet has proven me 100% correct.)
http://bsd.slashdot.org/story/13/02/16/2329259/netbsd-to-sup...
www.lua.org/wshop13/Cormack.pdf
Eventually Swift, Systems C# (if it ever comes out MSR), D, or any other language.
I'm afraid that these few CPU cycles do make a difference.
Throughput is the number one concern for thousands of network operators and service providers. They spend millions so you can get your content and not "509 Bandwidth Limit Exceeded".
You can't just go and replace all efficiency critical software by something that runs maybe just 10% slower.
Now, is this common? I have no idea. A great deal of 'C' code never sees the larger Internet anyway.
Edit: To clarify my point, "A fool with a tool is still a fool"
They - uhh I mean - we won't if it's impossible.
Back to my point: Tools can't replace knowledge. A fool with an tool is still a fool.
As in a Rust compiler will generate equivalent assembly as C in equivalent cases. With practically for now some places where Rust will win and some where C will win performance wise. But there are no practical reasons for Rust to be slower than C or C++.
What has gotten C compilers to generate the code quality that have nowadays is 40 years of work in compiler optimizations targeted to improve C compilers code generation.
If you think C++ is the answer to C's problems I think you suffer Stockholm syndrome. You've gotten so used to all the issues in C++ and started thinking about them as normal, so you stop seeing what a mess this language is.
By comparison there is no subset of C that lacks malloc/free.
The designers should have used traditional exceptions instead of trying to create something new --- the designers (just like Go's designers) had to go back and add exception support anyway.
I'd have liked Rust more if it'd focused on memory safety and less on being different for the sake of being different. I want a memory safe, non-GC language with traditional curly brace syntax, traditional types, and exceptional error handling. Modern C++ is a much better approximation of that language than Rust is.
For instance, I can happily "allocate" 100 terabytes of memory on my laptop (16 GB of RAM, 16 of swap):
use std::{thread, time};
fn main() {
let _v: Vec<Vec<u8>> =
(0..100_000)
.map(|_| Vec::with_capacity(1 << 30)) // 1GB each
.collect();
thread::sleep(time::Duration::from_secs(10));
}
Bare-metal programming, like a kernel, will typically use the lower-level `core` part of the standard library, rather than the whole thing (i.e. `std`) because the kernel will be literally providing the interfaces required for most of std-minus-core.
(I think it would be really great if the standard library could be less harsh when an OOM is actually detected, and I suspect it can be tuned in future, but it's not a show-stopper for most tasks.)
I strongly disagree. Handling memory allocation failure is trivial: throw an exception. It's also trivial to reserve enough memory to throw an exception. I honestly don't understand people who claim that recovering from OOM is futile because you need to almost always allocate memory to recover: in fact, you almost never do.
> Modern operating systems will over-allocate/over-commit memory
Windows is a very modern operating system and does not overcommit. (To its credit: overcommit is a terrible idea.) Even on overcommit systems, you can exhaust your address space and see allocation failures. No, 64 bits is not actually infinity. No, the whole world is not 64 bit. (Not every program even benefits from being 64 bit --- that's why the x32 ABI exists.)
> point is the allocations themselves never failed.
The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. It's very difficult to explain to people who work exclusively on overcommit systems that "memory" isn't a single resource, but a set of related ones, each of which can be exhausted.
> I think it would be really great if the standard library could be less harsh when an OOM is actually detected
The only reasonable thing to do on OOM failure is throw an exception, like Java and C++ do. Error codes are too cumbersome even if you dress them up and call them "monads" --- a lot of code allocates, so if you used error codes to indicate OOM, you'd have to manually propagate exceptions from almost everywhere) and just panicking or aborting is too brittle, since you can, in fact, successfully recover from allocation failure, persistent folk myths aside.
Now Rust has both Result _and_ panic/catch_panic, which I think is both sad and hilarious, as it's going to lead to decades of confusion over which error reporting mechanism to use in a particular situation. (The Go designers made exactly the same mistake, as did the designers of the old java.io.File interface.)
It's a shame for such a young language to have that kind of baggage already. If Rust's designers hadn't been so stridently anti-exception in the first place, we'd have ended up with a much better language.
The problem I'm talking about is allocation failure and OOM are not the same: handling out-of-memory reliably is a lot more than just checking for NULL in `malloc`, as I demonstrated. There's no guarantee that allocations will fail when they "should" and so you'll only learn that you're beyond reasonable limits when you touch enough pages that the operating system starts killing programs (possibly yours, possibly others), or when the user cannot actually use their computer in any reasonable manner.
> Windows is a very modern operating system and does not overcommit
Yeah definitely, it's why I was sure to mention the problems with swapping (sure, the memory is allocated and usable by the program, but in slow motion; not so great if you actually want to use the computer).
> The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory.
Yes, running out of virtual memory is one way for allocations to fail, but, as you say, there's a variety of different systems at play here. Only failure in one of those systems (requesting virtual memory/address space) is handled by checking for `malloc` returning NULL, and so focusing on just that is missing the forest for the trees. Presumably the overall goal is to handle memory exhaustion gracefully, not to check the return value of NULL (which is just one part of the former).
It's true that even with reasonable allocation failure reporting on a non-overcommit system, you can use too much memory and make life unpleasant for users without actually seeing allocations fail. Bounding resource use is important regardless. That's an issue of good behavior, however. Even absent good behavior, programs should at least be robust.
One of the reasons overcommit bothers me so much is that infects programming communities with the idea that resource consumption doesn't matter, since if you use too much, you're just going to get a SIGKILL anyway.
Unwind the stack and present an error message or take other appropriate action, just as we would for any other error. By unwinding the stack, we release owned resources, and some of those resources happen to be memory blocks. What makes memory so special?
> just finding all of your allocations is a nightmare in and of itself
Huh? In a non-GC system, if you can't find your allocations, you're leaking memory.
Memory is special insofar as it is a core resource that has so much contention that OOM situations may not be directly because of your application, and recovery may not even be possible without allocation.
I'm slowly coming to realize my argumentative point is losing ground -- this is great! :-)
> Now Rust has both Result _and_ panic/catch_panic
It even inspired a followup RFC for the purpose of codifying these restrictions in a brand-new fundamental trait: https://github.com/rust-lang/rfcs/pull/1323
> the inevitable
> the traditional exception model is simply the best one
You seem to be taking Rust's approach to error handling far too personally. :P It is not Rust's position that exceptions are a blight that must be scoured from the face of the Earth. It is merely Rust's position that there are easier ways to achieve concise and composable error handling for its intended domain, given Rust's other features and safety mechanisms.
It is, though. I'm not[1][2] the only one who very much prefers the exceptional model of error handling. Once the ability to catch exceptions exists, people like me are going to write exceptional code.
[1] https://github.com/rust-lang/rfcs/pull/1236#issuecomment-127... [2] https://github.com/rust-lang/rfcs/pull/1236#issuecomment-128...
> > the traditional exception model is simply the best one
> Citation needed. :P
When Java, Python, C++, and Common Lisp all converge on the same thing, maybe that thing has something going for it. Part of my claim is subjective, but I've worked on systems that used everything from abort to error codes to tagged unions to exceptions for error reporting, and I've found the exceptional code by far the easiest to work with.
In exceptional code, I can write foo().bar().qux() even though foo, bar, and qux can all fail, and without any extra syntactic noise.
> by disabling unwinding entirely and translating all panics to outright aborts
I believe that's a mistake: it's a reprise of the -fno-exceptions debacle of the C++ world, where the ecosystem is fragmented due to fundamental disagreements on the correct error handling strategy. You'll get into a situation where some components rely on panics being hard aborts for correctness and other components rely on non-aborting panics for error handling.
> And considering that Mozilla disables exceptions entirely in Firefox, I see no reason they would pressure the Rust developers to add exceptions to Rust.
Rust is probably a fine Servo implementation language. Tellingly, Mozilla also uses an infallible memory allocator. I merely believe that Rust's design choices for allocation failure and error handling lead to it being a losing contender when compared to modern C++ for general-purpose systems programming.
> Once the ability to catch exceptions exists, people like
> me are going to write exceptional code.
> You'll get into a situation where some components rely on panics
> being hard aborts for correctness and other components rely on
> non-aborting panics for error handling.
Except for the library authors who opt to write in C++ instead, of course. If you have to deliberately make a feature of your environment difficult to use, you're optimizing for the wrong thing. The Java experience shows that bad programmers will write bad code no matter how many safeguards you try to add to the language, so the only worthwhile "speed bumps" are those that help people who know what they're doing not accidentally shoot themselves in the foot. I cannot imagine an exception-catch block appearing in my code by accident.
Quite a bit of the discussion in the threads you mentioned has to do with flagging code as "requiring review" and adding "speed bumps" for "potentially troublesome" code. That's very worrisome, as it indicates that at least some members of the Rust community favor the Java-esque sandbox model of safety over the safety-catch model.
C++ has many flaws, but at least it never makes me feel like the language designers thought I need to be protected from myself.
> the only worthwhile "speed bumps" are those that help
> people who know what they're doing not accidentally
> shoot themselves in the foot. I cannot imagine an
> exception-catch block appearing in my code by accident.
(The irony in all this is that Rust's approach can be interpreted as an attempt to return exceptions to their roots by relegating unwinding to exceptional circumstances, as opposed to the routine error-handling solution that they've become in languages that have chosen to embrace exceptions (e.g. Python's egregious `StopIteration` exception for loop termination).)
Python does it right. I've never bought the argument that "exceptions are for exceptional circumstances". If you use error codes for some errors and exceptions for others, all you do is confuse people. The definition of "exceptional error" is subjective, after all.
As far as I can tell, the advice to use "exceptions for exceptional circumstances" originated in C++ circles in the 1990s, when support for exceptions was bad and throws were slow. The thinking was that you could efficiently use error codes for commonly-encountered errors and still get some of the syntactic benefits of using exceptions for rarer errors. Also, if you were using a compiler without exceptions support, you could turn your exceptions into aborts while not aborting just because a file doesn't exist. You can see vestiges of this thinking in the iostreams API.
Somehow, this performance hack morphed into commonly-accepted dogma. I don't accept this dogma. Exceptions are for any situation that a function can't meet its contract. In Python, if an iterator can't supply its next element, it can't satisfy the contract, and the exception tells you why. I don't think there's anything particularly wrong with that.
> instead of trying to create something new
monadic error handling isn't new. It's an old feature used by many languages, and preferred by many over exceptions.
I don't think it's useful to partition the world into microcontrollers and "LOL, memory is finite?" domains. Allocations can fail for normal programs, particularly when imposing subsystem-specific allocation limits or when running in small, crowded address spaces (e.g., mobile devices). It's often useful to be able to recover from this condition without completely killing the process, since process death is much more annoying for users than a specific task failing. (Wouldn't you rather see a video not play than see your entire program die?)
On a web server, 500? On a document editor? "Sorry, couldn't add attachment". What to do in response to error is application-specific. My point is that memory is not special and there is no reason to treat memory errors differently from other errors. You wouldn't abort if you run out of disk space.
> On most VM systems, malloc and friends oversubscribe what is actually available.
Windows is a common non-overcommit OS. Even on overcommit systems, it's possible to exhaust a process's virtual address space. (Overcommit systems also have ulimit -v.) If you do run out of either address space or commit, it's frequently the case that you can just stop what you were doing and display an out-of-memory dialog. The process of "stop what you were doing" can certainly free enough memory to keep going.
So Herb Sutter mostly agrees with your view, but my question is a corollary, and something he touches on: how do you keep going if you stop what you were doing? The process of recovering is not straightforward, and if you have threads running, that process is utter chaos, if not outright impossible in some languages. Even on non overcommit systems, the user finding another way to work around a memory failure is useless because the system is already hosed to the point where a small memory allocation will fail. Either way, the situation is so bad that the user does not have the ability to recover what was just done in your application without closing your application in the first place -- which, incidentally, leaves the user stuck in exactly the same place as if your application aborted.
The big idea here is that the OOM condition may not be because of your application.
You unwind. If you have threads, you reliably signal for them to stop (using resources you've set aside for the task), then you join the threads and continue unwinding. I think it's perfectly straightforward.
> Even on non overcommit systems, the user finding another way to work around a memory failure is useless because the system is already hosed to the point where a small memory allocation will fail.
Most serious operating systems have a way to say "run this process in this special environment where it can't allocate more than X MB of memory". (Some examples are NT Job objects, FreeBSD jails, Linux cgroups, and plain old ulimit.) In this kind of environment, you can certainly run up against limits for your process without the system having been rendered unusable.
> because the system is already hosed to the point where a small memory allocation will fail
Absolutely not the case. Imagine a state where a 100MB allocation (e.g., for an image) might fail (say, due to virtual address space fragmentation), but a 10 byte allocation would work just fine. The same principle applies to system commit considered globally instead of virtual address space considered per-process.
> the situation is so bad that the user does not have the ability to recover what was just done in your application without closing your application in the first plac
If that's the case, the application is just badly coded. Memory is not special. It's just another resource. If you can't keep track of resources and release them when they're unneeded (as they would be after an error), you're doing something wrong.
> http://www.gotw.ca/publications/mill16.htm
That article is factually incorrect. Sutter claims that "On...Linux, memory allocation always succeeds. Full stop." That's false, as a ten-line C program that calls malloc in a loop can demonstrate. You eventually run out of address space, especially in a 32-bit process. (There's a very popular overwhelmingly 32-bit Linux distribution out there called Android. You may have heard of it.) Sutter also claims that on other operating systems, overcommit is configurable. It's in fact configurable under Linux too. Linux can run in sane, non-overcommitting mode.
The title of the post talks about C, not C++.
It's 43 years old (times have changed), and while I'm sure C still has its places, I think it's time to come to terms with the fact that people suck at handling memory manually and making sure type conversions are safe & a host of other security risks.
Other, safer languages can accomplish many of the same things C does these days.
Again, I'm not calling for the end of C, but in a lot of situations, it's just asking for it now.
The fact they are also C libraries is because that was the only real option of the time when they were written; rather like saying most car accidents were caused by black ones when the only option was a model T.
Before C escaped UNIX there were other systems programming languages available.
Turbo Pascal and Apple Pascal ruled back then, at least in Europe. On Apple's case it was even the main OS language.
However with UNIX being adopted at work and universities, many wanted to be able to transport their work between home and work/uni. So people started to slowly adopt C.
I followed the same pattern with C++. Eventually Turbo Pascal wasn't no longer an option, so started using Turbo C++, C was just too primitive and unsafe already back then, vs TP.
At least C++ had some of the safety features I was used from TP, as long as one stayed away from C style coding.
With Pascal, I recall having to use more assembler.
I had to use Assembly in both cases.
I despair that "people suck at handling memory manually" :) It's not that hard, folks...
I just don't think C will be 'replaced' across the board though, but I suppose that's obvious.
Of the languages mentioned, I've only got experience with Go (which I use for APIs). Rust looks interesting though, I need to have a proper look at it.
Having automatic cars is convenient and has its uses. But the argument that not using a manual cars will cause less road accidents is misinformed. It looks like that because common sense dictates that as it's easier to drive, more people will drive it easier. But usually the result is that now you have people on road who could not drive a stick!
Some things are meant to be written in lowest level possible. Programming languages are tools and it helps us to have tools for entire spectrum of abstractions. Electronic screwdrivers haven't replaced manual ones in 2 decades. That's not how tools work.
As far as direct comparisons go, these isn't any indication that raising the level of abstraction actually helps with more secure, bug-free code. Compare Java code written (in FOSS setting) with C code written in last 15 years. I bet you'd find C code more secure, more fast, more faster to refactor than a lot of Java code. So if Java was a systems language, would the world be better off if it replaced C? Probably no. Which shows that it's our infatuation with new shiny toys (languages like Go) that's driving our opinion rather than evidence.
Yes, its true but it is also likely to cost your eyes.
Java FOSS code is often faster, more secure, and a lot easier to reason about than equivalent C code. This is not really a language thing but a standard library thing.
The use of char array with a null termination for human readable strings is a terrible 'C' ism. And glibc standard String struct would have saved us all a lot of (security) issues.
The second is a community thing. Java uses defined behaviour to drive optimisation. C tends to use undefined behaviour to drinve optimisatio. The C case really trips programmers up.
Most of these things are not actual technical but more social issues. Computers don't care, it is us humans who do.
But uses significantly more memory and requires a VM.
The VM probably is written in C, too.
Even if the intrinsic memory consumption of a java object is more than one for a C struct that does not mean it will in practice actually use more memory, or that it has too be that way due to unassailable theoretical grounds. See Realtime java for an example.
> The VM probably is written in C, too. Historical reasons only. It could have been fortran or pascal. Its not that C is magical in its low level. There are a whole bunch of other options as well. C was easy to get and it came with the OS for many developers when computers became more accessible due to lower costs in the late 70's and 80's. C is slow is not an uncommon opinion among fortran programmers, for good reasons. In practice the C++ parts of the different JVMs are getting smaller each release. And there have been JVMs that are written in Java with a bootstrap interpreter in ASM. As an OS with just java and assembly. After all, the language is turing complete...
Don't take me wrong Java is not some promised land without issues, its got plenty of those. I am just refuting the idea that just because it written in C it is fast claim.
GCC/XLC/ICC are not magic, neither is HotSpot or Vuze or J9 or any other JVM system.
In many ways C and Java are alike, crap languages introduced in a time when other, better alternatives where available. Only saved by industry adoption and decades of engineering work to make them fly.
Rust is so interesting as it has industry attention and is also academically interesting as bringing something new to the programming world. (at least I am not aware of any pl with the borrow checker concept worked out).
The reference JVM uses a mix of Java and C++, with code being migrated from C++ to Java and replaced with intrisics in each release.
Java 9 will bring Graal/Truffle plugins, which are a JIT compiler framework, written in Java.
Also I have real world experience with working with the same type of code in two different languages and seeing a clear difference. I've rewritten a lot of Objective-C code to Swift e.g. Objective-C as a language is much like C with similar problems as C. Just doing a straight conversion the Swift compiler picked up a lot of problems from my Objective-C code I had never discovered otherwise.
Lots of other people write about similar experience of getting significant bug reduction from moving to a much more type safe language like Swift.
The thing is, what a C programmer would consider a serious bug pertaining to performance and security, would not be fixable by Haskell being a better syntax. Ultimately it's the compiler that would bear the onus. Now generic compilers would always have performance issues and can't make everyone happy. That's the real problem here. It's easy to fix serious bugs (from perspective of systems programmers) in C than to fix them in Haskell. You have no control over compiled code or how it's going to turn out. The so called weirdness of C arises from the fact that non-seasoned C programmer do not understand why a C programmer is doing something. A weird C syntax usually is a signal to the compiler to write instruction code in a certain way.
It's not about correctness that a lot of C programmers are worried about. Well, at least not more than other languages. What we're worried about is what happens when rubber hits the road. It's very hard to do that from the driver's seat. Even Go, which tries to be closer to metal has issues around this. Things like Go, Obj-C, Swift are great where you simply want performance of machine code. But that's not the major argument to use C.
C isn't automatically easier to achieve high performance. Depends on what sort of problems we are talking about. If I remember correctly CouchDB was written in C/C++ originally and they switched to Erlang and got both much less code and higher performance.
But in something like a game engine were you want to be able to control memory layout and allocation C/C++ is a benefit. But Rust/Swift/Go allows you to do much of the same to various degrees.
At my work people write codecs and stuff and C is clearly a benefit due to its closeness to the metal. And as you say, yes they do write odd code at times because they know a lot about how the C compiler translates various constructs to assembly code. But this is specialist work. Most C coding doesn't involve this type of coding.
I think I just have to agree to disagree with you about correctness though. I had several years of C experience when I first tried Standard ML, and I manage to get a sorting algorithm correct on first try in SML, something I've never done in C. There will always be some stupid off by one type of error.
People have conducted studies of these sort of things and tend to measure better results for functional languages. One notable property of C programs is also that for any group of C programmers there is a huge variation in performance of the programs they write. The top C developers beat anybody. But average C programs are often slower than even average Python programs.
To take the car analogy further, whether your car is manual or automatic, seat belts and airbags are valuable. Neither of them compromises a driver's full control of their car.
Just use the right tool for the job given the circumstances.
well, for one the h/w model for one has far progressed beyond what the state of the art was during C's genesis. for example with the advent of multicore machines trying to use C's facilities to exploit available h/w resources becomes quickly unwieldy. witness the rise of event based approaches f.e. libevent and friends, or the rise of lockfree mechanisms etc. etc.