News for hackers. Stuff that matters.
Re:You can't Rust that by jnordwick on Saturday March 31, 2018 @02:21PM (#16724417)
I don't like the Things Move example. I'm not sure how true the general statement is (I'd never thought of it that way, but it isn't like how GC moves things around, and I'm not sure things are even more than in C++ -- I thought rust reduced unnecessary moves because would kill cache performance), but the example isn't entirely correct from my perspective.

Return values that fit into a register will be returned in a register, and his example is an 8 byte struct, so that returns in a register. Return values larger than a register will add an implicit first argument that is a pointer to memory where the return value should be written to. In that sense, it is very similar to C++ in that you are initializing into an allocated buffer.

As for "Refcounts are not Dirty", I would greatly disagree. Using refcounts to get around an overly aggressive borrow checker seems to be an ugly developing pattern in rust, and I feel they are giving away the performance many are fighting for by adding all these little inefficiencies to idiomatic rust. Add some refcounting here, add a Box or other indirection there, a chained monadic interface that can't short circuit and has to continually do error/null checks, etc... Soon it is death by a thousand papercuts. People fight hard for that extra 5% in performance only to have it taken away from them in interface and language issues.

Edit: Forgot about handles. Ugh. Completely unacceptable when you want to grow your tree data structure and you have to do a realloc and basically copy every node. If your tree is complete, then you are copying the whole tree every time you start a new level. The conclusions sound more like ugly hacks, than what you would properly design.

Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @03:35PM (#16724763)
> Using refcounts to get around an overly aggressive borrow checker seems to be an ugly developing pattern in rust, and I feel they are giving away the performance many are fighting for by adding all these little inefficiencies to idiomatic rust. Add some refcounting here

The performance hit isn't as it bad as it seems, because you only adjust the reference counts when you actually take an extra reference. This is different from shared_ptr in C++, because C++ automatically calls the copy constructor and so it's easy to end up with lots of reference count traffic.

Observe that, if we assume that the time spent in malloc() and free() dominates the time spent to adjust one reference count (which is a safe assumption), then the additional time overhead of Rc with a single owner is effectively zero.

> add a Box or other indirection there

Why do you ever need to Box unnecessarily in Rust? This is more of an issue with C++, where shared_ptr has an extra indirection and interior pointers and "new" encourage heap allocation.

I actually think that Rust in practice has the opposite problem: people are afraid to Box when they shouldn't be, causing unnecessary memcpy traffic.

> a chained monadic interface that can't short circuit and has to continually do error/null checks

How is this more overhead than in C?

> Edit: Forgot about handles. Ugh. Completely unacceptable when you want to grow your tree data structure and you have to do a realloc and basically copy every node. If your tree is complete, then you are copying the whole tree every time you start a new level. The conclusions sound more like ugly hacks, than what you would properly design.

Yeah, I would like to make a crate with an interface similar to petgraph but that actually mallocs every node separately. This should be readily doable.

I think the reason why nobody has made this crate yet is that copying the nodes on growth isn't as big of a concern in practice as it might initially seem, because the time spent in the growth case is amortized and made up for by extremely fast allocation of new nodes.

Re:You can't Rust that by gok on Saturday March 31, 2018 @04:45PM (#16725096)
> The performance hit isn't as it bad as it seems, because you only adjust the reference counts when you actually take an extra reference. This is different from shared_ptr in C++, because C++ automatically calls the copy constructor and so it's easy to end up with lots of reference count traffic.

Well only if you pass shared_ptr by value, which isn’t really idiomatic. But yeah the default is dumb in C++ and smart in Rust.

Re:You can't Rust that by therein on Saturday March 31, 2018 @03:04PM (#16724614)
The more I code in Rust, the more I reluctantly agree with this statement of yours. I wish you weren't right but I think you have a point. Something needs to be done about this before too much legacy is accumulated with this pattern.

> Using refcounts to get around an overly aggressive borrow checker seems to be an ugly developing pattern in rust, and I feel they are giving away the performance many are fighting for by adding all these little inefficiencies to idiomatic rust. Add some refcounting here, add a Box or other indirection there, a chained monadic interface that can't short circuit and has to continually do error/null checks, etc... Soon it is death by a thousand papercuts. People fight hard for that extra 5% in performance only to have it taken away from them in interface and language issues.

Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @03:39PM (#16724787)
> Something needs to be done about this before too much legacy is accumulated with this pattern.

Do you have any suggestions?

Re:You can't Rust that by therein on Saturday March 31, 2018 @06:15PM (#16725444)
I wish I did. Rust has a very interesting learning curve to say the least. It is a long fight with the borrow checker and it beats you until it teaches you its ways. Then you realize they were good practices all along.

And then comes a point where you realize that you need to write _some_ low level code that is unsafe. You read a lot about unsafe Rust and it almost allows you to almost work as if you are working with C. You can transmute a reference to native pointer type and then transmute it back as a mutable reference with a static lifetime if you want. But at that point you have lost the safeties and now you have to know exactly how to craft your transmute trickery such that you know there will be little to no overhead.

Perhaps it all comes down to the borrow checker being less strict about things. I can't tell exactly in which ways, perhaps about the lifetimes of references. Something doesn't feel as natural to reason about working in Rust as it does in C but I can't put my finger on it.

One thing I can say though is I wish I didn't need to use lazy static for non-primitive static globals.

But overall I really like the general direction that the language is heading. Love the way dependencies are managed although TOML wasn't the best format choice in my mind. crates.io, docs.rs and the entire community and the ecosystem are great and unique too.

Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @08:07PM (#16725876)
I'm confused about how it's hard to craft transmute trickery to eliminate overhead. Transmute should be zero-overhead...
Re:You can't Rust that by therein on Saturday March 31, 2018 @09:34PM (#16726203)
It is the mental and the syntactic overhead. For a simple reference to say my framebuffer can become more complicated than they should need to be.
Re:You can't Rust that by anewhnaccount2 on Sunday April 01, 2018 @02:53AM (#16727112)
For exposing a safe subset of transmutes, there's https://github.com/nabijaczleweli/safe-transmute-rs

I don't like it's API at the moment, but perhaps it could be turned into something nicer?

Re:You can't Rust that by burntsushi on Sunday April 01, 2018 @10:03AM (#16728301)
Please don't use that crate. From what I can tell, it is not actually exposing safe routines and gets things wrong. I've filed issues in the past, and I've just now filed a couple more.
Re:You can't Rust that by comex on Saturday March 31, 2018 @08:15PM (#16725911)
Not the parent, but I think that fixing interior pointers - i.e. something like `rental`, but built into the language and more ergonomic - would make a big difference, and also make Rust feel less restrictive to C/C++ programmers even in cases where there's a reasonable workaround.
Re:You can't Rust that by Manishearth on Sunday April 01, 2018 @12:15AM (#16726680)
> Using refcounts to get around an overly aggressive borrow checker seems to be an ugly developing pattern in rust

Every single large, "modern", C++ codebase I've worked with uses way more refcounts than is typical in Rust.

(and it's worse because C++ refcounting patterns involve more refcount churn due to how they're designed on the copy constructor, and shared_ptr uses atomics even when unnecessary. Though these large codebases tend to use a slightly different design that lets them decide atomicity per-type)

Re:You can't Rust that by the_mitsuhiko on Saturday March 31, 2018 @02:29PM (#16724459)
If a value returned from a function actually moves or not is currently up for Rust to optimize. It's not something you can depend on.

About the refcounts: since the counting is explicit (calls to clone()) they at least in my experience don't really show up. Most of the refcounted objects I deal with bump the refcounts once when some task spawns and decrements it when it ends. I have yet to see refcounts to change in hot code paths.

//EDIT for your edit:

> Edit: Forgot about handles. Ugh. Completely unacceptable when you want to grow your tree data structure and you have to do a realloc and basically copy every node.

Sure, but that's not my point anyways. At any point you can fall down to writing unsafe code and building a safe abstraction on top of it. This is to help developers not run into walls. I don't think that handles are the best thing invented but I don't think "well you can't do that in Rust until we some time in the future" and not provide an alternative is a particularly good suggestion.

Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @02:59PM (#16724591)
But if they do... oh boy, what fun can be had by spamming ooeratins on the same memory location from multiple cores! If this isn't thrilling enough, try doing that with atomic operationsto be extra thread safe. I had some really fun experiences with this in C++. The best one was a factor 100 slowdown by activating compiler optimizations.
Re:You can't Rust that by berkut on Saturday March 31, 2018 @03:04PM (#16724616)
Are Rust ref-counts atomic? (I'd assume so).

Atomic ref-counts can actually have a surprising amount of overhead in many multithreaded scenarios, so much so that going back to raw pointers over shared pointers in C++ for some use cases of passing pointers around can provide a huge win.

Re:You can't Rust that by therein on Saturday March 31, 2018 @03:10PM (#16724649)
There is Rc and Arc, the latter being atomic. If you want to pass them between threads, you'll want an Arc, and in fact you will want an Arc<Mutex<Data>> so your data becomes:

  let data = Arc::new(Mutex::new(Data::new()));
And if you want a mutable static global that's accessed across threads, you'll need something like:

  lazy_static! {
    // 0 -> None
    // 1 -> Circles
    // 2 -> Bezier
    static ref DRAW_ON_TOUCH: Arc<Mutex<u32>> = Arc::new(Mutex::new(0));
  }
I love Rust but it is making easy things harder than they need to be.
Re:You can't Rust that by dbaupp on Saturday March 31, 2018 @05:28PM (#16725261)
How is safely accessing a mutable global from multiple threads an easy thing?

Also, the Arc isn't needed in lazy_static: just a mutex is enough for mutability, the sharing (which is what Arc controls) comes from being a static.

Re:You can't Rust that by comex on Saturday March 31, 2018 @08:19PM (#16725935)
And if there's just a single u32 inside, there's no need for a whole mutex, you can just use a zero-overhead AtomicU32. …or you could, if it were stable.
Re:You can't Rust that by retrocat on Sunday April 01, 2018 @01:53AM (#16726959)
AtomicUsize is stable!
Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @03:16PM (#16724672)
Hm, this distinction between non-atomic and atomic ref counting sounds like a scat-gun sized footgun waiting to go off when it is least expected. Is there any protection for non-atomic ref counting in a multithreaded environment in Rust?
Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @03:37PM (#16724773)
> Is there any protection for non-atomic ref counting in a multithreaded environment in Rust?

Of course. Rust is memory safe. The compiler will prevent you from sharing non-thread-safe reference counted objects between threads.

This is a significant advantage over C++, where the compiler offers no such protection and so the typical solution is just to use atomic reference counting (shared_ptr) everywhere.

Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @03:44PM (#16724811)
So are you claiming that the exact same solution is good in rust and bad in C++?
Re:You can't Rust that by moosingin3space on Saturday March 31, 2018 @04:26PM (#16725008)
I'm not pcwalton, but he is correct. The Rust compiler only allows objects that implement the Sync trait to cross thread boundaries. Rc<T> is not Sync, while Arc<T> is. Sync is a marker trait, with impls automatically generated by the compiler for types that are comprised entirely of Sync types, or manually (and unsafely) specified by the author. As a result, Rc<T> cannot be misused from multiple threads without unsafe.

To learn more about this, look here: https://doc.rust-lang.org/beta/nomicon/send-and-sync.html

Re:You can't Rust that by the_mitsuhiko on Saturday March 31, 2018 @03:45PM (#16724820)
I'm not the person you are replying to but I would never use a non atomic refcounted smart pointer in C++ because the risks associated with it are just too high. In Rust however the compiler protects me from misuse.

Separately emotionally I feel different about Arc<T> because all the counting is explicit and I see it. Understanding where shared_ptr<T> in C++ counts however can be quite a challenge, even if you consider yourself proficient with C++.

Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @03:57PM (#16724871)
I may be in the minority here, but if you are programming properly, your understanding of the code should always be good enough that this is a non-issue. If you manage to trigger a compiler error like this at all, you are most likely hacking, not developing.
Re:You can't Rust that by oconnor663 on Saturday March 31, 2018 @04:27PM (#16725012)
This sort of mistake gets harder and harder to prevent as programs get larger. For example, one programmer might use a non-atomic Rc pointer to hold a string in some random Error value. Later, another programmer might try to parallelize that part of the program, without realizing that sending an Error across a thread boundary is now illegal. Or maybe the same mistake happens in the other order.

Here's Niko Mastakis talking about a case where that happened to him (I think): https://youtu.be/lO1z-7cuRYI?t=4m

Re:You can't Rust that by swsieber on Saturday March 31, 2018 @06:56PM (#16725608)
I don't trust myself to not make mistakes though even when I have a thorough understanding of what I'm making.

"... trusting the programmer is fundamentally wrong, no matter the programming language"

Re:You can't Rust that by the_mitsuhiko on Saturday March 31, 2018 @04:03PM (#16724913)
Should probably but I'm not a good enough developer (yet?) to guarantee that. I rather take the extra safety from the compiler.
Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @03:59PM (#16724888)
Reference count problems routinely result in vulnerabilities. Google for [cve reference count].
Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @04:10PM (#16724940)
But these vulnerabilities can only happen if reference counting is not automatic, right?
Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @04:28PM (#16725016)
No, because of the way references in C++ work—you can take a reference to the inside of a reference counted object without manipulating the reference count.

They could also happen if you used non-thread-safe reference counting on a thread-safe object. Now this seems to be rare in practice, because non-thread-safe reference counting is unusual. If it became more prevalent, however, I could see it happening.

Re:You can't Rust that by empath75 on Saturday March 31, 2018 @04:24PM (#16725005)
Real programmers apparently never make mistakes.
Re:You can't Rust that by moosingin3space on Saturday March 31, 2018 @04:31PM (#16725024)
Unofficial slogan of the C apologism task force, apparently.
Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @04:38PM (#16725060)
No. Real developers design their data structures and algorithms carefully. When it is tine to implement them, everything should be laid out clearly enough. There are industries where this kind of process is required and it truely shows in the quality of the results.
Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @08:09PM (#16725882)
I mean, that's great in theory, but we've been trying the "just design the program better" approach for 40 years and have been failing.
Re:You can't Rust that by kbenson on Saturday March 31, 2018 @05:09PM (#16725194)
If the vast majority of software is not made by real developers (and I think by the definitions you are espousing it is not), then I think one of the definitions is in need to change.

As beneficial to your argument as it would be for that change to be the definition of "software", I don't think that's likely.

Re:You can't Rust that by gmueckl on Saturday March 31, 2018 @10:58PM (#16726449)
I needed to make a distinction in the dedinition I gave. And I stand by it, even if the wording comes across as snarky. There are software engineering methods that are proven to work. They do not involve just diving in and changing the code. But I also think that there are valid reasons to not apply these rigid development practices to certain kinds of projects. For the rest, we might well get mandatory development standards forced on us to

I am well aware of the pressure on developers to get requested feature done quickly. But the vast majority of bugs that get shipped would be easily prevented by more thorough processes. All those projects writing highly reliable software demonstrate this. But the price tag has a few zeroes appended as a result.

Re:You can't Rust that by pohl on Saturday March 31, 2018 @11:56PM (#16726621)
There are software engineering methods that are proven to work.

Rust’s type system is arguably one of them.

Re:You can't Rust that by gmueckl on Sunday April 01, 2018 @06:45AM (#16727658)
I was talking about processes, not tools. Rust is just tool. But good software comes from thorough processes.
Re:You can't Rust that by kibwen on Sunday April 01, 2018 @10:35AM (#16728484)
Good software comes from processes that aren't afraid to adopt tools to automatically enforce those processes in order to guard against human oversight.
Re:You can't Rust that by gmueckl on Monday April 02, 2018 @05:14AM (#16733278)
Tools come from processes. You need to know what the tool needs to do (the process) before you can write it.
Re:You can't Rust that by pohl on Sunday April 01, 2018 @10:59AM (#16728595)
Type systems are applied mathematics. You can’t get more engineering-y than that.
Re:You can't Rust that by gmueckl on Monday April 02, 2018 @05:17AM (#16733281)
See my reply to the other comment.

Type systems are tools that simply the process of checking that you do not mangle data by applying the wrong instructions to them. They didn't come out of nowhere.

Re:You can't Rust that by pohl on Monday April 02, 2018 @02:25PM (#16737571)
Math is a tool, too, yet applying mathematical methods is one of the defining characteristics of engineering.
Re:You can't Rust that by wtetzner on Saturday March 31, 2018 @06:24PM (#16725491)
> There are industries where this kind of process is required and it truely shows in the quality of the results.

It also shows in the cost of building the software. Most software companies aren't willing to pay the cost of that process. Like it or not, you will almost certainly interact with software written by these companies, either directly or indirectly.

Re:You can't Rust that by the_mitsuhiko on Saturday March 31, 2018 @03:19PM (#16724687)
> Is there any protection for non-atomic ref counting in a multithreaded environment in Rust?

Yes. It errors at compile time (`std::rc::Rc<i32>` cannot be sent between threads safely)

Re:You can't Rust that by pcwalton on Saturday March 31, 2018 @03:38PM (#16724777)
> Are Rust ref-counts atomic? (I'd assume so).

No, not unless the object is actually shared among multiple threads.

Re:You can't Rust that by the_mitsuhiko on Saturday March 31, 2018 @03:09PM (#16724642)
Arc<T> is atomic, Rc<T> is not.
Re:You can't Rust that by quickben on Saturday March 31, 2018 @03:13PM (#16724667)
Imagine your example in a singleton pattern scenario. It would be quite a performance hit.