I work on a C codebase that does this, although in the slightly weaker sense that it does drop the abstractions at a few isolated interaction points with external APIs (think openssl, linux system calls, and not a whole lot else). Yes, there is quite a lot of NIH. With essentially-uniform use of checked data structures, and an extremely comprehensive suite of automated tests getting run under ASAN (originally Valgrind), memory safety errors almost never get so far as being committed to the main branch. This is a complex, >1M SLOC distributed system that has seen several years of production use at this point, and as far as I can recall we have not seen a single memory safety related issue in production (a few have managed to to get as far as certification testing). General resouce-leak class issues have struck a few times, but are also pretty rare.
Proprietary, naturally, so I can't actually show you (sorry), but it absolutely can be done in practice. It isn't even really all that difficult, it just needs to be done from the start, and then you just need a bit of discipline to keep it up.
And more power to you. Note the beginning of the parent comment, however:
> Just because you 'can' write such an array implementation doesn't mean you will
So yes, even if the codebase you work on does have these 'mythical', hard-to-achieve properties, that doesn't mean that most or even many C codebases will.
Good engineering entails observing what problems actually occur and working to fix those. Memory safety issues do commonly occur in C codebases. Regardless of whether the fix in C is simple or even trivial, programmers aren't doing it. So, Rust has some value because it forces the programmer to produce code that is largely free from this type of issue.
Enforcing norms like 'be more disciplined when writing C' or 'stop using external libraries' is much harder than simply using a different language.
>
> […] it just needs to be done from the start, and then you just need a bit of discipline to keep it up.
Here's a neat idea: wouldn't it be cool and save a lot of time if the compiler did this for you automatically, from the start?
Of course, you say it's easy to do it manually, but something tells you your company might have needed to pay less for development if the compiler did it automatically with no human intervention required.
Yes, but when the project started the only existing compiler that met all requirements was C (also C++, although that was not chosen, by reasoning I disagree with). We are in a domain where we derive material benefits from the low-level control C gives us (we have a bunch of highly specialized memory management and I/O), and are not willing to accept GC pauses. There's a common sentiment that we would have used Rust if it had existed when we started, but it didn't so we didn't and so it goes.
Rust, at least in this regard and probably others too, is no better than C, and for me it isn't enough to justify the horrible and complex syntax.
Rust checks at runtime and panics if your program exceeds the bounds. You can opt-in to asking if the bounds are exceeded and fail gracefully if you like, or if you want to promise the compiler you know for sure your bounds are tight, you can use unsafe blocks and act like C. Opt-in to danger.
C lets you do it with no checks. You have to opt in to the safe path of checking and failing. You don't automatically segfault if you exceed the bounds; instead you read arbitrary memory. Welcome to the land of undefined behavior. You may crash, but more likely, you will read some value from an unexpected place, and carry on executing incorrectly for who knows how long. Opt-in to safety.
That's what people mean by Rust is safe by default. And that's just bounds checks. Carry that notion over to pointers, references, threads, lifetimes, ...
What ever made you think "Safe Rust" meant "compile time checks of runtime values are possible" or "C is just as safe because it lets you index outside an array"?
Optimizers already try to prove index bounds to eliminate unnecessary checks, and static analysis tools to demand necessary checks. Turning the latter into compile time errors is a reasonable approach if your language can provide sufficient information to deal with false positives - likely by forcing you to add your own bounds checking to explicitly handle out-of-bounds cases.
A language John Carmack was using or researching at one point comes to mind, which had this kind of thing going on IIRC. I'm afraid I can't find it off hand, so I might recall in error.
And the point of this thread is that Rust's default (check index bounds and fail at runtime unless the check can be proven unnecessary by the compiler or optimizer) is safer than C's default (don't check anything by default and hit UB if the index is out of bounds).
GCC and clang both have sanitizers either built in or available for them. Sure, it's not default, but let's not act like there is no choice in C but to account for every OOB access while programming or to read memory you don't want to.
Furthermore, I never said that compile time checks of variables are possible, but rather we could move to using dependent typing, or at least a way to judge whether a variable would work as a subscript based on the type of the array and variable.
The Rust designers didn't do that. Instead they put in a feature common in the two most popular C compilers to "panic" at runtime instead of accessing memory. That's nothing. It's rubbish. And if you know to "catch" the panic, why don't you check the value of what you're subscripting with? Saying you can catch the panic is missing the point of unintentional OOB accesses, which is that they're unintentional.
C is just as safe with regard to OOB accessing, and to be honest that's pretty poor in 2017.
If you are using gcc or clang, you have more options. True. But not all C compilers give you those options. However, the point is moot, since I never said you can't catch these things in C; I said it wasn't the default. Which you agree with.
> I never said that compile time checks of variables are possible, but rather we could move to using dependent typing
You didn't say anything about dependent typing. You said "Rust is no better than C". And I'm pointing out that it is. Dependent typing may be even better in some cases; I'm not arguing otherwise.
> Saying you can catch the panic is missing the point of unintentional OOB accesses, which is that they're unintentional.
No one said you should catch the panic. You can use Vec::get() for example if you are using runtime-derived indices and want bounds checking in an ergonomic fashion.
And saying a panic for unintentional OOB is the same as in C is not true, since you get a panic by default in Rust, and to get one in C, not only must you be using a specific compiler or two, you must have the sanitizers enabled for every source file in your program. Not "by default" by any stretch.
> C is just as safe with regard to OOB accessing, and to be honest that's pretty poor in 2017.
It is nowhere close, and saying it is is pretty poor in 2017 as well.
And you are still ignoring pointers, references, lifetimes, threads, ... you know, the other things that also help Rust make "Safe by default" and C "dangerous by default".
https://doc.rust-lang.org/std/primitive.slice.html#method.ge...
C will actually read arbitrary memory, Rust won't, that's the difference.
We're talking about a situation where the length of the array is not statically known and you access it out of bounds at runtime. Rust checks first if it's out of bounds, and if it is, DOES NOT blindly read the memory anyway (as C would) but exits.
I don't understand how you could think the two situations are at all equivalent.
Why no malloc? It was originally due to the small memory sizes for code and data. The less standard library the better, and dynamic allocation may lead to heap fragmentation and a subsequent crash when malloc fails.
And it's not limited to the VM itself: check out npm "native extensions" like `json`. Not to mention glibc, or the OSes themselves.
By your definition, nothing is safe. And you're right ;)
I note that Firefox is using some Rust code now - so perhaps that will change at some point, for at least one of the common JavaScript implementations, in the not too distant future. I don't imagine we'll see it for the majority within the decade - but who knows, maybe I'll be pleasantly surprised.
I have less hope for the widespread adoption of OS kernels written in safer languages - given the general unwillingness to even use C++ there (although plenty of toy/'research' kernels in safer languages do exist.) Although maybe we'll see one within the next century? Perhaps a microkernel for use in containers?
Of course, that still leaves bugs in the JITs, compilers, hardware, 'legacy' native interop, unsafe{} blocks, ...
Maybe
I think currently there's no plan for it. Maybe after they finish servo to the degree where it supports all modern html features
That one "if" is (by definition) not zero-cost.
> C++ implementations obey the zero-overhead principle: What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better.
Two points:
What you don't use, you don't pay for: if you don't use array indexing, you won't get a bounds check. In addition, you can call an access method without a bounds check as well, so it truly is only if you use the checked version.
What you do use, you couldn't hand-code any better: that bounds check is written the exact same way you'd write it in C.
Therefore, this is a zero-cost abstraction.
The Haskell `newtype` example I gave was meant to illustrate this, as newtype's are respected by the type system and then are treated as the underlying type at runtime.
Most code doesn't use bounds checking, because the branch is a safety net you should never hit, even in theory. Any code that does hit it is already broken. Correct programs using bounds checked indexing will in general be slower than but equivalent to a program where indexing instead results in undefined behaviour.
"You couldn't hand-code any better", well, I won't argue on that point, as it sounds contentious. ;)
_Should_ never hit is very different than will never hit...
My point is that Bjarne Stroustrup wasn't comparing against writing the exact same program the exact same way. He was comparing against what you'd get if you dropped down to ye olde C or Assembly and wrote the same algorithm there, without redundant work or waste.
The comparison shouldn't be the language's GC versus SteveGC, it should be the language's GC versus an ideal, manually implemented allocator. Equally it shouldn't be built-in bounds checked indexing versus manual bounds checked indexing, it should be built-in bounds checked indexing versus an ideal, manually implemented indexing scheme. If you want safety against out-of-bounds, it seems to me the ideal method would be a proof, not runtime overhead.
I don't know of a single language that comes with a GC that does this, do you?
> He was comparing against what you'd get if you dropped down to ye olde C or Assembly and wrote the same algorithm there, without redundant work or waste.
Right. I agree with this.
But basically, we are arguing over an extremely fine semantic, which is "should you even want bounds checks in the first place." If you don't, then don't use a method that has bounds checks. The one that does will have them. They'll both cost the exact same as writing it in C or assembly.
To put it another way, let's say I was a C++ developer on the fence about Rust. If I read this conversation, I'd see that indexing gets called "zero-cost" despite the overhead. Since tons of things in Rust are "zero-cost", like traits, closures, borrowing, etc., all of those things now have doubt cast on them. How can I really trust that these things are actually getting compiled efficiently?
If instead the conversation pointed out that this was one of a few cases where safety took priority over truly being zero cost, but that there were tools in place to mitigate the cost (iterators, unsafe indexing, LLVM), I'd have a much more positive outlook that focussed on what Rust did right.
That said, I can appreciate focusing on other things when talking about the principle; I only brought them up here because we were literally discussing them. I think there's much better examples when actually attemping to convince someone.
Java, .NET, Go, ML and Lisp compilers.
Escape analysis allows to do that, even if just in certain special cases.
Plus the more one uses value types and less heap, the GC needs to work less, specially if we take languages like Modula-3 into this mix.
Escape analysis may not use the GC for those variables, but it is still a pervasive runtime cost in both senses unless it's totally gone.
The problem isn't the language it's the developers.
The "heartbleed in rust" example is a great one, and it arises in real life in many high level language APIs for file I/O and sockets. You have an allocation, and you have a count of available bytes coming back from a read() function which may be lower than the allocation size. So you are creating a "virtual" array bound from nothingness. Fail to respect it (without bounds checks) and you will see bugs.
If you reject that this is a valid way to write code, maybe in your API every read() style function will always return the correct size enforced by your JVM or whatever, but you will do too many allocations and over-tax the GC.
If you accept that this makes sense, then you must embrace a more C style way of thinking, where array bounds are created and destroyed at will and must be enforced through your own actions... And suddenly you see the other side of this coin, which reflects valid and true things about the universe, that you may want to chop up a buffer into multiple pieces - and that's OK.
(Now, I wouldn't be surprised if Rust has mechanisms to chop up arrays in the way I describe and enforce the bounds you provide it... Which would be handy. But frankly does not completely destroy the validity of the C approach or substitute for a proper understanding of it. Without that understanding, you will code more heartbleeds.)
[T]::split_at is probably what you're looking for.
Almost all array handling in Rust is done through slice types which are tagged with sizes.
However, you can't bungle the creation of a slice in rust without using explicitly marked unsafe code.
New languages here and there every day. Replace this replace that. When, in the end everyone is simply reinventing the "wheel" over-and-over.
All these languages end up as assembly.
I honestly don't know who I'd put my money on between "AI takes over the world" and "programmers stop writing buffer overflows"
The CVE database. Just because you 'can' write such an array implementation doesn't mean you will, doesn't mean your third party libs will, doesn't mean any of your legacy code uses it, and certainly doesn't mean you will properly test said array implementation correctly.
The number of mitigations added to C compilers and OSes dealing mostly with C and C++ code. ASLR, W^X, /GS, -fstack-protector-all, AddressSanitizer, ... - note the lack of similar tools, or demand for them, for, say, JavaScript - despite it enjoying a similar ubiquity.
I ask this in bad faith: I encourage you to share a single nontrivial codebase which actually creates the abstraction you've described and religiously adheres to using it throughout. As to why this is in bad faith: I'm definining "nontrivial" here to mean using 3rd party APIs - which will operate on C style arrays, not your project specific safe wrappers - and thus by definition won't be "religiously" sticking to said abstractions when using said APIs. By these definitions, the codebase I'm asking for doesn't exist - by definition. Even relaxing the "third party" rule, I haven't actually worked on a nontrivial C or C++ codebase without buffer overflow problems.
Now, e.g. Rust will have the same problems when interacting with C APIs - and nontrivial programs will end up doing so eventually. However, by virtue of the language itself embracing safe-by-default, you're less likely to run into the same problems when consuming Rust APIs.
You can also use third party static analysis tools to ensure you're using a "safe C subset" (such as MIRSA C), but "nobody" does that.