News for hackers. Stuff that matters.
Re:Rust Memory Management by pjungwir on Tuesday May 09, 2017 @12:21PM (#14301315)
As someone whose career has been mostly in Java/Python/Ruby but is also comfortable with C, learning Rust has been like a step function, where each time I get stuck I correct some discrete misperception and then have an easier time. I wish I had written these down, but here is one that helped me a lot and is also a little funny:

When Rust folks talk about "move", it is not just jargon about ownership. They actually mean move to a new place in RAM. If you could do a printf("%p", foo) before and after, it would be different. So obviously if there are other pointers to that object, they aren't going to be valid anymore. Figuring this out really helped me reason about what was allowed and what wasn't. Maybe it will help out someone else too. "Oh, you mean move!" :-)

Re:Rust Memory Management by khuey on Tuesday May 09, 2017 @12:41PM (#14301548)
> When Rust folks talk about "move", it is not just jargon about ownership. They actually mean move to a new place in RAM. If you could do a printf("%p", foo) before and after, it would be different.

That's not really true though. In an optimized build rustc will elide memcpys whenever possible, which it often is. "Move" in Rust really is about ownership.

Re:Rust Memory Management by pjungwir on Tuesday May 09, 2017 @12:50PM (#14301633)
That is true, but personally I think about that as coming "after" all the ownership stuff checks out, and as an implementation detail that I can usually ignore. I do keep it in the back of my mind though.

Btw I have a saying: when you teach, at first you have to lie a little. You paint the broad strokes, then you refine, then refine again, etc. So it's a question of what is a helpful mental model. Knowing the optimization behavior is helpful for some things, but for figuring out valid memory behavior it's useful to pretend you don't know about that.

Re:Rust Memory Management by pjungwir on Tuesday May 09, 2017 @02:23PM (#14302602)
Another breakthrough I recall was understanding that there are three alternatives: move, copy, borrow. I don't know about the docs nowadays, but when I was starting it seemed like explanations would always pick two of these and describe the differences in a binary way. That made things hard to retain, because it was like the spatial layout in my head was always shifting. Establishing for myself that there were three was very helpful early on.
Re:Rust Memory Management by sevensor on Tuesday May 09, 2017 @10:41AM (#14300393)
This is the clearest, most concise explanation of these issues that I've seen. Certainly not comprehensive, but coming from a C perspective it answers about 95% of my questions. This may be the post that gets me actually writing something in Rust.
Re:Rust Memory Management by leshow on Tuesday May 09, 2017 @02:06PM (#14302455)
I'm surprised. I read through it and thought it wasn't very good. I've been writing Rust here and there for some time now. It seemed like there were a lot of errors in the post.
Re:Rust Memory Management by zeenix147 on Wednesday May 10, 2017 @11:03AM (#14308711)
I'm the author of that post. I'm sorry to hear that my post wasn't useful and contained errors. I'd be very much interested in hearing what the errors are, so I can correct them. I actually tried all the examples so I'm guessing the samples at least are correct.
Re:Rust Memory Management by sevensor on Wednesday May 10, 2017 @10:42AM (#14308558)
Well, I'm not in a position to see the errors, since I've not attempted anything serious in Rust. This post pretty much aligns with my expectations for what I would hope Rust does, given my experience with C. I'll be disappointed if I find that it's misleading me! Thanks for the warning!
Re:Rust Memory Management by zeenix147 on Wednesday May 10, 2017 @11:17AM (#14308822)
Thanks. I'm glad my post was useful to at least some folks. :)
Re:Rust Memory Management by jayflux on Tuesday May 09, 2017 @10:45AM (#14300433)
Very good, enjoyed that, was very clear and easy to follow. For anyone who's hungry for more I recommend the Rust Book Second Edition (not finished yet) https://rust-lang.github.io/book/second-edition/index.html

I hope he does one for lifetimes as I still find them confusing

Re:Rust Memory Management by zeenix147 on Wednesday May 10, 2017 @11:16AM (#14308815)
Thanks. Haha, me too. :) I'll try to write one when I feel comfortable enough with lifetimes myself.
Re:Rust Memory Management by falcolas on Tuesday May 09, 2017 @09:53AM (#14299976)
> stack allocation is preferred over the heap allocation and that's where everything is allocated by default

Has this ever been proven to be a problem in practice, given that a program's stack is typically limited in size? Would the recommended workaround (if this was a problem) be to change variables to use heap instead?

Re:Rust Memory Management by dom0 on Tuesday May 09, 2017 @10:06AM (#14300077)
Stack allocation is the default in all practically used languages that actually have a stack (e.g. C, C++), and heap allocation is explicit in all of those as well.

Allocation more or less everything on the heap (i.e. using the stack only for calls and returns) is something that's done out of necessity in exclusively GC'd languages, like Java or Python.

Re:Rust Memory Management by bogomipz on Tuesday May 09, 2017 @10:33AM (#14300317)
>"Stack allocation is the default in all practically used languages that actually have a stack (e.g. C, C++), and heap allocation is explicit in all of those as well."

That was my first thought as well when I read that passage. Do you know why the author might have felt the need to state this specific in the context of Rust then?

Re:Rust Memory Management by paupino_masano on Tuesday May 09, 2017 @10:44AM (#14300431)
C# and Java allocate objects to the heap and primitive types to the stack by default so perhaps it may be in context to that (i.e. It's not explicit but rather implicit on the type being used).
Re:Rust Memory Management by DorothySim on Tuesday May 09, 2017 @10:49AM (#14300470)
> C# and Java allocate objects to the heap and primitive types to the stack by default (...)

Note that it only applies to local variables. If you have a primitive inside an object then it's allocated on the heap as part of the outer object.

Re:Rust Memory Management by bogomipz on Tuesday May 09, 2017 @04:03PM (#14303352)
I see, that would make sense. Thanks.
Re:Rust Memory Management by falcolas on Tuesday May 09, 2017 @10:25AM (#14300242)
Sure, it's the default, but C and C++ encourage have patterns where calls to `malloc` and `new` much more common than in Rust. Especially since strings are almost always handled as malloc'ed buffers.

I'm just wondering if the extra emphasis on stack allocation has created issues; this article doesn't cover the allocation of Strings or Vecs (wrapping them in RC and ARC boxes), and most other references on the net are a few years old now.

Re:Rust Memory Management by masklinn on Tuesday May 09, 2017 @10:46AM (#14300445)
> I'm just wondering if the extra emphasis on stack allocation has created issues; this article doesn't cover the allocation of Strings or Vecs

As in C++, these are small stack structures (a few words, IIRC it's 3 for Vec, and String is backed by a Vec) pointing to a heap-allocated buffer.

Re:Rust Memory Management by steveklabnik on Tuesday May 09, 2017 @10:51AM (#14300488)
The second edition of the book goes into it in great detail https://doc.rust-lang.org/nightly/book/second-edition/ch04-0...
Re:Rust Memory Management by pjmlp on Tuesday May 09, 2017 @02:31PM (#14302662)
Languages like D, Oberon or Modula-3, just to name three out of many, do have GC and also support explicit global and stack allocation.
Re:Rust Memory Management by dom0 on Tuesday May 09, 2017 @02:42PM (#14302769)
exclusively GC'd
Re:Rust Memory Management by pjmlp on Tuesday May 09, 2017 @02:52PM (#14302849)
I saw that, but wasn't sure how you actually meant it.
Re:Rust Memory Management by johncolanduoni on Tuesday May 09, 2017 @11:09AM (#14300673)
Java uses the stack for local variables (including pointers to objects on the heap and primitives), and I believe Python uses a stack for the same as well. Return pointers are the least of stack usage.
Re:Rust Memory Management by Veedrac on Tuesday May 09, 2017 @12:27PM (#14301381)
They don't allocate objects on the stack, though; they merely store bindings (and, in Java's case, primitives).
Re:Rust Memory Management by pcwalton on Tuesday May 09, 2017 @01:21PM (#14301977)
> Has this ever been proven to be a problem in practice, given that a program's stack is typically limited in size? Would the recommended workaround (if this was a problem) be to change variables to use heap instead?

It is occasionally a problem. For example, when writing my old emulator, I statically allocated the system's RAM (which has a fixed size) on the stack at first before thinking better of it and moving it to the heap. But such situations are uncommon.

Re:Rust Memory Management by partycoder on Tuesday May 09, 2017 @01:59PM (#14302399)
It's great to see Gnome adopting Rust. Vala was OK, but we all Vala compiled to C rather than binary.

I think the move to Rust should attract more developers to the Gnome ecosystem. The learning curve of C or Vala + GTK + GObject and related tech can be a bit steep.

Re:Rust Memory Management by Symmetry on Tuesday May 09, 2017 @11:18AM (#14300746)
I'm not sure I understand why, in design terms, add_first_element doesn't return ownership of the vectors given that they don't seem to be passed in mutably?
Re:Rust Memory Management by pornel on Tuesday May 09, 2017 @11:34AM (#14300879)
It's by design - this syntax intentionally expresses such contract.

As a matter of policy Rust doesn't try to figure out behavior of functions from their bodies, because that would mean changes in the implementation could accidentally break the API.

While in this example taking ownership is pointless, there are cases where it's meanigful: e.g. a method that closes a file handle would take ownership of the handle to ensure that the handle can't be used anywhere else after it's closed.

In Rust ownership allows mutability (if you own an object, and there are no other references to it, you can do whatever you want to it).

Re:Rust Memory Management by Symmetry on Tuesday May 09, 2017 @11:44AM (#14300978)
Wouldn't closing a file handle be an action requiring mutability? If the file handle is just a integer then it would be copyable and ownership wouldn't come up as an issue meaning discussions of safety would be moot.

I understand not trying to look at function bodies but the mutability of a function seems to be strictly a property of it's type signature.

But I think what really bothers me is that whether a value is actually passed to a function by pointer or by copying it onto the stack is governed by whether it has the 'copy' trait - not something obvious to the casual reader of the program. Silently converting what looks like a pass by value call to an in-practice pass by reference based on a trait definition elsewhere in the code base just seems excessively magical to me and I'd be much happier if the compiler threw an error at

let answer = add_first_element(v1, v2);

because pass by value requires the copy trait which Vecs v1 and v2 don't have.

Re:Rust Memory Management by masklinn on Tuesday May 09, 2017 @11:48AM (#14301008)
> Wouldn't closing a file handle be an action requiring mutability?

Closing a file requires ownership and is actually an RAII action: when the file object itself goes out of scope (which by definition means all extant references have been removed) the handle is closed.

> But I think what really bothers me is that whether a value is actually passed to a function by pointer or by copying it onto the stack is governed by whether it has the 'copy' trait - not something obvious to the casual reader of the program.

That's not correct. The Copy trait means it's semantically copied rather than moved so the original value can still be used, it does not change the technicalities of parameter passing which is always by-value (unless you're explicitly creating a reference) although things may then get optimised differently.

> Silently converting what looks like a pass by value call to an in-practice pass by reference

There's no such thing. Both Vecs are passed by value. Outside of method calls (which have some syntactic sugar) Rust requires references to be created explicitly.

> I'd be much happier if the compiler threw an error at

> let answer = add_first_element(v1, v2);

> because pass by value requires the copy trait which Vecs v1 and v2 don't have.

It has no reason to and it does not. `add_first_element` gets its argument by value, since Vec is not a Copy type v1 and v2 are thus moved into the function and not accessible anymore from outside it.

Re:Rust Memory Management by Symmetry on Tuesday May 09, 2017 @12:38PM (#14301507)
>Closing a file requires ownership and is actually an RAII action: when the file object itself goes out of scope (which by definition means all extant references have been removed) the handle is closed.

Ah, no wonder I misunderstood then. I'd forgotten that RAII was something Rust had and that feature does very nicely explain that design decision.

I suppose the reason a Vec doesn't implement Copy then is that its members might not, like a file handle?

Re:Rust Memory Management by masklinn on Tuesday May 09, 2017 @12:45PM (#14301592)
> I suppose the reason a Vec doesn't implement Copy then is that its members might not, like a file handle?

That is correct, Vec has a pointer to a heap allocated buffer, meaning if Vec were Copy, you'd get two Vec structures for the same backing buffer and that's a recipe for disaster.

Re:Rust Memory Management by pornel on Tuesday May 09, 2017 @11:58AM (#14301101)
> Wouldn't closing a file handle be an action requiring mutability?

In Rust there is no distinction between mutable owned data and immutable owned data. There are only mutable/immutable references and bindings, but if you own the data, you can mutate it.

   fn foo(v: Vec)
   fn foo(mut v: Vec)
are the same function. `mut` is not part of the type. Even the first function can do `let mut v = v` and make immutable owned `v` a mutable owned one (since Rust can guarantee there are no other references to it, so the mutation is not observable from outside).

It's different from `&mut Vec`, which is a distinct type, and it mutably borrows the Vec.

> If the file handle is just a integer

That's why in Rust such values are wrapped in "newtypes" that don't allow copying, e.g. `struct Handle(i32)`.

> pass by value call to an in-practice pass by reference

Passing ownership of the `Vec` doesn't copy the items.

Ownership/references and passing by value/reference are different concepts.

`Box<Foo>` is an owned value, but it's implemented as a regular pointer (`Foo* ` in C). `&[Foo]` is a reference, but it's actually a small struct passed by value (`struct Ref {size_t len; Foo *data}` in C).

Re:Rust Memory Management by Veedrac on Tuesday May 09, 2017 @12:23PM (#14301336)
Copy does _not_ affect compilation, only type checking. If you add Copy to a type, the generated code will not change. If you remove Copy from a type, the code will either stop compiling or be exactly the same.

The difference between a copy and a move is whether you can still use the old value, and this is checked in the type system.

Re:Rust Memory Management by idsout on Tuesday May 09, 2017 @11:26AM (#14300804)
One reason that comes to my mind is more control over resources. You can be sure the memory allocated for the two vectors in main is freed in add_first_element.
Re:Rust Memory Management by Symmetry on Tuesday May 09, 2017 @11:35AM (#14300889)
But if ownership was returned then it could be freed in main. And because it was allocated on the stack in the first place there doesn't seem to be any reasonable way for the callee to, say, return a reference to an element of the vector because then the caller could return that reference again but then you'd have a value still referred to by the outside context sitting where the stack should be able to grow so I assume that Rust would disallow this and force you to make a copy of anything you want to return.