It's also like, a perception thing in some sense. Imagine someone writes some code. They get a compiler error. There are two ways to react to this event:
"Wow the compiler didn't make this work, I have to think about memory all the time."
"Ah, the compiler caught a mistake for me. Thank goodness I don't have to think about this for myself."
Both perceptions make sense, but seem to be in complete and total opposition.
The Rust community has done laudable work in bringing down the cognitive threshold of "manual / compile-time" memory management, but I think we're finding out that the returns are diminishing quickly and there's still quite a chasm between borrow checking and GC with respect to developer velocity.
This seems pretty close to objective. It doesn't seem like a semantic question at all. These things are all "knowable" and "catalogable".
(I like Rust more now than I did last year; I'm not dunking on it.)
Maybe it depends on other factors too. But in practice, I basically never think about memory management. I write code. The compiler sometimes complains. When it does, 99.9% of the time I go "oh yeah" and then fix it. It's not a significant part of my experience when writing code. It does not slow me down, and the 0.1% of the time when it does, it's made up for it in some other part of the process.
I wish there was a good way to actually test these sorts of things.
I can also get user feedback sooner and thus pivot my implementation more quickly, which is a more subtle angle that is so rarely broached in these kinds of conversations.
The places where I think the gap between Go and Rust is the smallest (due to Rust's type system) are things like compilers where you have a lot of algebraic data types to model--Rust's enums + pattern matching are great here.
To take two GC'd languages, I'm proficient in both Java and Scala. It usually takes me a little longer to write something in Scala, but when I'm done, I've almost certainly written fewer bugs in the Scala program than the Java program (I've also written many fewer lines of code, but that's another topic).
For me, it's the type system that helps the most. Given that Rust's type system is much stronger and expressive than Go's, I do expect to write fewer bugs in Rust than in Go. But it does feel like, if I had more experience with Go, I'd be significantly faster writing Go than Rust. (Then again, the more I write Rust, the fewer write-compile-fail-fix cycles I have to go through, and the compiler's ability to accept code as safe improves pretty frequently.)
Still, though (and I know this isn't the question at hand, but...), I personally value greater chances of correctness at compile time way more than development speed. While some types of bugs can be a fun adventure to track down and fix, most bugs I encounter are some mix of boring and annoying. I honestly would prefer to spend 2 weeks building and 2 days debugging over 1 week building and 1 week debugging. I really do find debugging that annoying. (Fuzzy numbers; I don't actually think I'd build 2x as fast in Go as Rust.)
In my experience, I don’t get much additional correctness for the extra effort, but rather I get independence from the GC, which is worth much less to me.
If we’re optimizing for correctness alone, I think development times could improve significantly by swapping the borrow checker for a gc. I know the borrow checker aids in correctness beyond what a gc does, but IMO the returns diminish rapidly. And I’m not sure how well this would work in practice, but maybe you could keep the borrow checker and add a GC, with every reference type being gc<> by default (not sure if that would recoup any of the extra correctness that a borrow-checker affords or not).
> I know the borrow checker aids in correctness beyond what a gc does
Really depends on what kind of programs do you write. I found that my Rust development gets slowed down only because I have to spend time to create the proper types. Memory management and lifetime problems are very few in my practice (but I can agree that they can swallow a time -- only when you are new though).
Are there some examples of that?
Control is primarily exerted over consumers of your API rather than the actual resources. This can be enforced through a combination of Drop implementations, and closures / lifetimes; the classic example is Mutex's LockGuard. In a GC language (eg Go) they give you defer or finally blocks that can accomplish the same thing, but that is always optional and up to other programmers to remember to do. Compare: you can't typically make someone run destructors in a GC language; you also wouldn't be able to guarantee the destructors have run at any particular point in time.
The one area you have more control over actual resources is knowing when memory is freed. Some people need to know when memory is freed, because they have allocated a lot and if they do it again without freeing, they'll run into trouble. To know for sure, simply use a normal owned type or a unique pointer (Box); when it goes out of scope, that's when its destructor is run. No such feature exists in a GC language, because you can never know at compile time when nobody else holds a reference.
As a thought experiment: in JavaScript with WebAssembly, an allocation in WASM can be returned to JS as a pointer. You need to free it, somehow. Can you write a class that will deallocate a WASM allocation it owns when an instance of the class is freed by the JS GC? (Answer: no! You need a new language-provided FinalizationRegistry for that.)
Maybe D people would have something to say about this as well, but I'm not a D person. What you're describing doesn't seem impossible in D to me, though.
Dynamic-extent appears to be more similar to the "register" hint in C than to anything in Rust, in that it's an implementation-defined-behaviour hint. Rust has no such thing as hinting at storage class. Your variables are either T (stack) or Box<T> (heap) or any other box-like construct involving T. You maintain complete control at all times, nothing is implementation-defined, and it's explicit. You can implement (and people have implemented) dynamic switching between stack and heap storage in a Rust library.
https://lib.rs/smallvec (stack to heap), https://lib.rs/tinyvec (smallvec with no unsafe code), https://lib.rs/arrayvec (stack only)
As you can see, these three library authors get to control very precisely how their types allocate and deallocate, and you basically mix and match these and the stdlib's smart pointers (and Vec) + other libraries like arenas, slot maps, etc to allocate the way you want.
> you'd probably achieve that type of resource control by exposing only WITH- macros in your API*
Yes, this and similarly using with_* closures both work, but both are more limited than destructors that run when something goes out of scope. A type that implements Drop can be stored in any other type, and the wrapper will automatically drop it. You can put LockGuard in a struct and build an abstraction around it.
What I think people _should_ focus on with Rust versus Go (et al) is that Rust allows you to choose where you _place_ memory. You can choose the stack or the heap. The placement can matter in hot regions of code. Additionally, Rust is pretty in-your-face when it comes to concurrency and sharing memory across thread/task boundaries.
In general, if it's a datastructure where you have to use pointers, you'll have them Box'ed, but you would try to avoid that if you can. In your example of a heap, you'd want to use an array-based implementation, probably backed by a growable Vec, and use indexes internally. A peek function would still return a normal Rust reference to the data, and the borrow checker would make sure that you don't mutate the heap's backing array while that reference was still in use, etc.
One thing I would add that you need to be wary of destructors with large pointer data structures in Rust since it can easily stack overflow. When using Option<Box<T>> you need to be careful to call Option::take on the pointers in a loop to avoid stack overflow.
struct Node {
elem: i32,
next: Option<Box<Node>>,
}If you use a &mut Node as your pointer, you are attempting to answer those questions with "not me". Someone else has to own the Nodes. They've got to be somewhere on the stack or on the heap. There's nothing stopping you from defining the next pointer as Option<&'a mut Node>. The problem is actually constructing a list.
Not many answers tell you why you can't do this in practice. I agree that this is not explained well enough in general, because new Rustaceans don't intuitively reach for references so it probably doesn't come up much and they're hard to use for this. But it's not that hard to see why:
Imagine you try to allocate all the Nodes at once (e.g. an array), and then use &mut references into the pre-allocated array. In order to set one &mut Node's next pointer, you will have to hold another &mut Node to set it to. This means you need to acquire mutable references to array elements, in the order that you wish them to appear in the linked list. This is actually really tricky to do: slice::get_mut(index)'s returned reference borrows the entire slice, so it doesn't let you have a &mut reference to two nodes at the same time. You need smaller &mut [Node] slices, somehow.
slice::split_first_mut is one way (in order), but if you have an array and can only create a linked list in the order nodes appear in the array, what's the point? Just use the array! Any other compiler-checked access order scheme will also be so limiting that you should just use a data structure of that exact shape anyway. To use an arbitrary order, you're going to need unsafe, so you'd basically be writing C.
To be fair, there is basically one application of this, and it's to have a sparsely populated constant size array that needs to be iterated in order. I made a demo:
https://play.rust-lang.org/?version=stable&mode=debug&editio...
The other problem is that you can't resize the backing array: your &mut Node references would be invalidated.
For this reason, pre-allocated lists like these are usually done with indices instead of references. The overhead is one pointer + offset and then a bounds check when dereferencing.
---
The other solutions answer the ownership question like so:
- You can use Box, so that each Node (acting as a list head) owns the entire tail of the list, uniquely, such that no other list can also refer to it. The tail is freed when the head is, unless of course you detach it first (let tail = node.next.take();).
- You can Arc/Rc the nodes, so that each node has a pointer, but not a unique pointer, to the next node. These can be duplicated, so lists can exhibit structural sharing if you are comfortable with that. Because of the sharing, freeing the head does not necessarily free any/all of the tail.
If you actually want this intrusive linked list functionality, consider using a real-world implementation like this: https://lib.rs/intrusive_collections
Use channels whenever possible.
https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s...
Yes we had RAII, but it didn't work for a lot of cases where we needed it.
Rust = when writing your code, you explicitly describe the ownership and lifetime of your objects and how your functions are allowed to consume/copy etc. them and get safety as a result
C = when writing your code, you explicitly allocate and free your objects and you get no assistance from the language about when it is safe to copy/dereference/free/etc. a pointer/allocation
For trivial cases, kind of. But once you start to do anything remotely sophisticated, no. Everything you do in Rust is checked w.r.t. memory management, but you still need to make many choices about it. All the stuff about lifetimes, borrowing, etc: that's memory management. The compiler's checking it for you, but you still need to design stuff sanely, with memory management (and the checking thereof) in mind. It's easy to back yourself into a corner if you ignore this.
Yes Rc and equivalents have a performance overhead, but for many use cases the overhead really isn't that bad since you typically aren't creating tons of copies. In practice, I've found one can ignore lifetimes in almost all cases even when using references except when storing them in structs or closures. So really you would just need to increment the Rc counter for structs/closures outside of allocation/deallocation which is dominated by calls to malloc/free.
(Normally I use subclassing in Qt to associate extra state with a widget, but gtk-rs's subclassing API was arcane and boilerplate-heavy. Perhaps there's alternative paradigms for state management that follows Rust's single ownership principle better. Some people take a React/Elm-style approach, but I don't think virtual DOMs and diffing the entire UI tree on each user interaction are the last word on GUI interactivity and updates, and I don't find the added memory of virtual DOMs and CPU of generating/diffing them acceptable, but rather "pure overhead" to be eliminated in favor of minimal targeted UI state updates.)
The trick is to pre-allocate all your objects and buffers and reuse them in a ring buffer. Similar techniques work in zero-malloc embedded C environments.
Similarly, idiomatic modern C++ rarely needs new and delete calls, but I'd still say it has manual memory management.
I suppose it's reasonable to talk about degrees of manual-ness, and say that memory management in Rust or modern C++ is less manual than C, but more manual than Go/Python/Java.
In GC languages, memory management is generally runtime through the interpreter/runtime. In C, memory management is generally done at programming time by the (human) programmer. In Rust, memory management is generally done at compile time by the compiler. There are exceptions in all three cases, but the "default" paradigm of a language informs a lot about how it's designed and used.
From what I've read, Rust memory management actually requires more work but provides fantastic safety guarantees. This could mean that rust actually lowers productivity at first, but as the complexity of the code base grows, some of that productivity is restored or even supercedes C/C++ because you spend no time chasing runtime memory bugs.
For some products or projects, the costs of shipping a security flaw caused by a memory bug exploit could be high enough that a drop in productivity from Rust relative to C is still more than justified due to external costs that Rust mitigates.
> PRO-REWRITE: Rust has manual memory management, so we would avoid the problem of having to wrestle with a garbage collector because we would just deallocate unused memory ourselves, or more carefully be able to engineer the response to increased load.
> ANTI-REWRITE: Rust has manual memory management, which means that whenever we’re writing code we’ll have to take the time to manage memory ourselves.
Isn't part of the point of Rust that you don't manage memory yourself, and rather that the compiler is smart enough to manage it for you?