News for hackers. Stuff that matters.
Re:Two Years with Rust by modernerd on Monday March 23, 2020 @08:04AM (#22663178)
Everyone always seems so positive about Rust. I'd love to try it for some personal projects. Are there any downsides beyond the niggle the author mentioned?

Are compilation speeds an issue for anyone?

Is there much that can be done to improve this? (Both in Rust itself and at a developer level; presumably a faster dev machine helps?)

Re:Two Years with Rust by kouteiheika on Monday March 23, 2020 @09:02AM (#22663672)
> Is there much that can be done to improve this?

I have a 50k+ lines of code project which usually recompiles after a change in ~2 seconds in release mode.

There are many tricks which can be used to improve compile times to the point that even on medium-sized projects the compile time is not an issue. But you need to keep a certain discipline to adhere to these.

1) Use LLD instead of the system linker.

2) Don't add dependencies willy-nilly. Especially for trivial stuff which you don't need pedal-to-the-metal optimized. (e.g. do you really need to add that 4000 lines long SIMD-optimized base64 encoder/decoder, or can you live with a naive 10-lines long version you can write yourself in a few minutes?)

3) Feature-flag gate dependencies/features not necessary during development. (e.g. do you actually need HTTPS support during development, or can you test your webapp on HTTP and only compile-in the TLS stack for production deployment?)

4) Avoid heavy dependencies. (e.g. there are some popular web frameworks for Rust which have over 100+ dependencies by default; if you pick such a framework then your compile times are obviously going to be very heavily affected)

5) Use dynamic dispatch (&dyn T and Box<dyn T>) instead static dispatch (impl T) when accepting generic arguments in cases where you don't need pedal-to-the-metal performance.

6) If you absolutely need to use static dispatch purely for ergonomics (and not because of the performance) then create two function - a dynamically dispatched private one which accepts a &dyn T and contains the actual functionality, and a public one which accepts impl T and is a one-line wrapper around the private one.

7) Don't use #[inline] annotations if you don't absolutely need them.

8) Split your project into multiple crates.

Re:Two Years with Rust by leshow on Monday March 23, 2020 @12:36PM (#22665664)
> 6) If you absolutely need to use static dispatch purely for ergonomics (and not because of the performance) then create two function - a dynamically dispatched private one which accepts a &dyn T and contains the actual functionality, and a public one which accepts impl T and is a one-line wrapper around the private one.

Would you really recommend this as common advice? It seems like not a good idea to me. If all you cared about was binary size and compilation speed, maybe, but not otherwise. Same with blanketly recommending use of &dyn T instead of <T>. There are other problems with dynamic dispatch in rust, namely it's kind of a pain if you need to `+ OtherTrait` with it.

Re:Two Years with Rust by kouteiheika on Monday March 23, 2020 @01:25PM (#22666293)
Yes I would. In general people tend to overuse static dispatch even when it's not really necessary. Of course the issue is a little bit more nuanced than "always use X unless Y" and there are tradeoffs in play here that need to be balanced.

For example, if your function is really small - yeah, it's probably fine to just use static dispatch. If you're writing a generic data structure - you most likely also want it to be a statically dispatched Struct<T>, but with a healthy dose of #[cold] annotated non-generic functions for the cold paths. However, let's say that you have a function that accepts a filesystem path and loads a PNG from it - you do not want the PNG loading code to be 1) duplicated in every compilation unit (compilation time bloat), and 2) monomorphised three times just because you passed an `&str` once to it, a `String` another time, and a `PathBuf` yet another time (compilation time and executable size bloat), so you definitely do want dynamic dispatch here (at least under the hood with the two function trick).

I do think the default should indeed be &dyn T and you should only go for impl T when you can actually clearly substantiate why you should use it, instead of the other way around which is the default now in the Rust ecosystem. (Which is how you end up with 20+ second edit-compile cycles one of the sibling comments mentioned.)

Re:Two Years with Rust by aw1621107 on Monday March 23, 2020 @06:27PM (#22669468)
> you do not want the PNG loading code to be 1) duplicated in every compilation unit (compilation time bloat), and 2) monomorphised three times just because you passed an `&str` once to it, a `String` another time, and a `PathBuf` yet another time (compilation time and executable size bloat), so you definitely do want dynamic dispatch here (at least under the hood with the two function trick).

It appears that there is some work to (partially?) address this problem in PR #69749 [0], which, from what I can tell, aims to avoid instantiating functions on unused type parameters.

It's not a perfect solution for your case, but should be fairly easy to take advantage of.

[0]: https://github.com/rust-lang/rust/pull/69749

Re:Two Years with Rust by eximius on Monday March 23, 2020 @03:23PM (#22667714)
Could a proc macro be written that uses dynamic dispatch in debug and static dispatch in release? That would be optimal for dev compilation speed and binary speed, at the cost of binary size. It seems like a pretty good tradeoff for most cases.
Re:Two Years with Rust by kouteiheika on Monday March 23, 2020 @04:27PM (#22668374)
IIRC I think there was a crate with a procedural macro like that which did something kinda similar to this. However currently that isn't really optimal for quite a few reasons: 1) having a procedural macro by itself pulls in extra dependencies, 2) it introduces extra work for the compiler so it does negatively impact the compile times (the procedural macros don't operate directly on the AST, so they have to parse the token stream into an AST, process it, serialize it back into a raw token stream, and the compiler has to parse it again), 3) Rust's current procedural macro machinery doesn't yet support emitting proper error messages from a procedural macro, so if you'll get an error or a warning from a piece of code generated by a procedural macro it will just point to the #[name_of_the_macro] annotation instead of the actual location where the issue originated from.
Re:Two Years with Rust by leshow on Monday March 23, 2020 @04:27PM (#22668380)
Your practice deviates significantly from what other people are doing in the ecosystem, but if you like it for your personal projects, more power to you.
Re:Two Years with Rust by kouteiheika on Monday March 23, 2020 @06:37PM (#22669536)
> Your practice deviates significantly from what other people are doing in the ecosystem

Yeah, I know, that's what I said in the last paragraph! (: And that's also why I enjoy a 2 second recompile times (I could probably go even lower with some more refactoring) in release mode (debug mode is too slow for me at runtime) for a medium sized project while other people are struggling with half a minute compile times on projects less than half of mine's size.

I'm not saying that what I wrote is the ultimate panacea, but it's certainly one way of tackling the problem of compile times and executable bloat, and it'd certainly be nice for more crates in the ecosystem to take this issue more seriously. (You can only do so much by making the compiler faster.)

Re:Two Years with Rust by modernerd on Tuesday March 24, 2020 @07:09AM (#22673826)
Thank you, this is very helpful.

> 1. Use LLD instead of the system linker.

A note for macOS users that unfortunately LLD is not supported reliably there yet:

https://github.com/rust-lang/rust/issues/39915

https://users.rust-lang.org/t/llvm-lld-linker-fails-on-mac-o...

From https://lld.llvm.org/

> The linker supports ELF (Unix), PE/COFF (Windows), Mach-O (macOS) and WebAssembly in descending order of completeness.

Re:Two Years with Rust by giovannibajo1 on Monday March 23, 2020 @09:03AM (#22663681)
I’m writing a largish (~15k) hobby program in Rust. I’ll focus on negative points, there are many positive.

Compilation time is terrible, I need something between 20 and 40 seconds for edit-compile-cycle (MacBook Pro 2016, i7, 16gb ram) which kills productivity for me especially since as a hobby I tend to have small windows. I should probably dedicate time to investigate how I can speed it up (split it in more crates than naturally required, refactor how tests are written, etc) but this is a turn down by itself.

Productivity is still low. Even after writing so many lines I still can’t feel productive; I still face problems writing code that compiles, and am forced to workaround issues with partial borrowing of structures, or similar issues. This happens anytime I need to do something “new” form an architectural point of view (writing a new “component”), or if I attempt something “smart” (eg, trying to refactor to reduce duplication). This matches what this article said: Go is far more productive for me. If the architecture is fixed and I just “write the code in the right place”, then I can get decent speed (modulo the compile time issue).

Re:Two Years with Rust by cesarb on Monday March 23, 2020 @10:03AM (#22664173)
> Compilation time is terrible, I need something between 20 and 40 seconds for edit-compile-cycle (MacBook Pro 2016, i7, 16gb ram)

A trick available on newer releases of rustc is "cargo check", that stops the compilation before the code generation (which is the slow part). If all you need at the moment is to see whether it would compile (no syntax errors, no wrong types, no borrow check errors), it can help a lot.

Re:Two Years with Rust by staticassertion on Monday March 23, 2020 @03:13PM (#22667610)
I also work on a ~15kloc rust project. I use cargo check all the time. Still extremely slow, not to mention that my whole system can be locked up by rustc, which really kills development.

I'll probably find a way to cgroup rustc just to deal with this issue.

Re:Two Years with Rust by pimeys on Monday March 23, 2020 @04:33PM (#22668440)
I solved this by building myself a 3950x workstation. 16 cores and 32 GB of RAM does miracles for Rust workflows.

That and the guidance kouteiheika gave above puts you far already.

Re:Two Years with Rust by pjmlp on Monday March 23, 2020 @05:25PM (#22668940)
Which means other system languages will continue to thrive for developers not so lucky with Rust compiler rigs.
Re:Two Years with Rust by pimeys on Monday March 23, 2020 @06:00PM (#22669277)
The funny thing is I paid about the same from my rig that people pay for a Macbook Pro. It is not really unaffordable and you get a great development machine.

Of course nowadays the combination of Emacs and rust-analyzer gives you a very fast and nice workflow. That and one terminal running

  cargo watch -s 'clear; cargo check --tests --color=always 2>&1 | head -40'
works miracles.
Re:Two Years with Rust by staticassertion on Monday March 23, 2020 @04:52PM (#22668621)
Yeah I have 8 cores(HT) and 16GB, which is apparently just not enough.

I think my plan is to just move builds to some kinda EC2 instance that I can bulk up.

Re:Two Years with Rust by pimeys on Monday March 23, 2020 @04:57PM (#22668661)
It's probably just the thermals. A desktop CPU runs circles over any laptop due to tremendously better thermals and power window.
Re:Two Years with Rust by staticassertion on Monday March 23, 2020 @05:04PM (#22668729)
It's possible, but my laptop runs fairly cool - it's elevated on a stand and is 18 inches, with a lot of ventilation. Of course it doesn't compare to a desktop system, but even still, this thing is hardly weak. It's a bit absurd to require 16 cores and 32GB of RAM to be productive.
Re:Two Years with Rust by leshow on Monday March 23, 2020 @12:26PM (#22665536)
If you divide the project up into sub crates, using a "workspace" you can improve compile times. If you're not using that feature already. Not saying that compilation speed isn't an issue, but that's one thing that can help.

Sibling comment mentioned `cargo check` which is pretty fast too. If you use an editor with a decent rust plugin like vscode (using RLS or rust-analyzer). It will run `cargo check` by default for you.

Re:Two Years with Rust by dpbriggs on Monday March 23, 2020 @08:30AM (#22663384)
If you're trying to do something very specific that doesn't align with Rust's ownership/memory model it can be quite challenging.

For example, I wanted to make a skiplist library with features not present in other crates. Trying to go the totally safe route was a cognitive overload, and trying to express complex invariants in that form was painful. Completely doable though, if you're willing to learn it all.

I eventually settled on a much more unsafe, but very granular approach that's probably not going to get a good rating with cargo-crev.

Re:Two Years with Rust by probably_wrong on Monday March 23, 2020 @10:30AM (#22664427)
I want to echo this point. I tried to write an implementation of an automata-based algorithm in Rust, and my main takeaway was that Rust really doesn't like linked lists. There is currently a popular book [1] that focuses entirely on how one would write such a thing.

That's not to say that it can't be done - I believe there's a crate implementing the same algorithm I was going for. But my main point is: if you see "systems programming" and think "oh, like C", then you are going to have a difficult time.

[1] https://rust-unofficial.github.io/too-many-lists/

Re:Two Years with Rust by dpbriggs on Monday March 23, 2020 @10:47AM (#22664584)
Totally agree. Anything that touches multiple owner territory gets ugly pretty fast. You end up with a variety of Cell objects, Rc::weak pointers, etc. Again it's possible but not trivial to figure out.

Now I wish I read that resource before writing the library, as I would have saved a considerable amount of time (~30h / 2k lines of rust). Just using raw pointers (NonNull) everywhere meant comments like this [0] but was far more elegant and understandable overall.

It was spooky seeing rust segfault before everything was ironed out + valgrind + miri tested. But it's also really nice to have the borrow checker figure out sketchy lifetime extensions / iterator invalidation for you.

[0] https://github.com/dpbriggs/convenient-skiplist/blob/d781954...

Re:Two Years with Rust by quietbritishjim on Monday March 23, 2020 @01:50PM (#22666654)
Rust newbie here.

If you're working on a project like that, how bad (or even possible) would it be to just stick unsafe around almost everything, if the alternative is to use C++? I mean, unsafe Rust is still no worse than C/C++, right?

Re:Two Years with Rust by probably_wrong on Monday March 23, 2020 @05:26PM (#22668947)
In my case, I wouldn't do it for two reasons.

First, if the whole point of using Rust is "memory safety", then writing everything unsafe would be missing the entire point. I'd rather do it well at some time in the future, when I properly understand the language.

Which leads me to my second point: at this point, I'd rather do it in C (which I did). I haven't done much C lately, but I'm at least aware of its strengths, weaknesses, and best practices. Better to use an old tool well than a new one badly.

More expert Rust programmers can probably give a better answer.

Re:Two Years with Rust by winrid on Monday March 23, 2020 @12:42PM (#22665751)
This is a good point. You'll have lifetime symbols everywhere, for example.
Re:Two Years with Rust by pjmlp on Monday March 23, 2020 @09:43AM (#22664011)
Yes, I can hardly use my travel netbook for Rust hobby coding as pulling new dependencies will keep me waiting for 20 minutes on average.

My hobby coding on C++ is never so slow, thanks to binary dependencies. Even source based package managers like vcpkg allow to stage binary libraries, so you just have to compile the world once, and can share with the team, or whoever you feel like it.

Then C++ compilers, not only support incremental compilation (which Rust also supports), they also have incremental linking, even at function level.

So unless you are doing some crazy metaprogramming, even C++ can be faster to compile than Rust.

What they can do is having cargo support binary libraries, and also have some form of JIT/AOT integration, like many other toolchains allow for.

Interpreted/JIT for development and AOT for release mode, for example.

Re:Two Years with Rust by dgb23 on Monday March 23, 2020 @11:04AM (#22664768)
Disclaimer: I'm a big fan of Rust and I'm very hopeful about its future. This is why I spend a lot of time studying it.

As someone who is learning and comparing the language there are things that I think definitely suck about Rust:

* Syntax is ugly, unclean and overloaded. There are things like the ? operator which seem to have no purpose other than sugar in the sense of "you write a bit less code", which I find to be useless, obscure and even had restricting implications on async/await. Also the whole language suffers from being a weird mix of C-like statement syntax and FP-like expression syntax, which takes time to get used to. Function signatures are both verbose and arcane. I wish Rust just admitted early on, that it needed a different way of expression and went from first principles instead of superficially trying to mimic other mainstream languages.

* Rust being a very error driven language is a good thing, but make it less suitable for prototyping the happy path. This is fine but a constraint that one should keep in mind.

* powerful pattern matching, dataless programming with traits, FP-like API on iterators and option/result etc. make it very expressive on a higher level, but more basic things like string operations and array indices (possibly other things as well) are far less ergonomic than in other modern languages. Typically in other/older languages you'd struggle with abstraction but the basic stuff is easy, in Rust you struggle with the basic stuff and abstractions are ergonomic.

* all around (a bit) too much emphasis on structs over hashmaps as the default data structure for associative grouping.

Things I like about Rust but are not for everyone:

* Rust is very error driven, you tend to write a lot of code around errors, options, results.

* pattern matching/destructuring is the main way to write branching logic.

* a lot of emphasis on enums (tagged unions) to express configuration, state, variants and things like that.

Re:Two Years with Rust by hechang1997 on Tuesday March 24, 2020 @01:34AM (#22672259)
> Also the whole language suffers from being a weird mix of C-like statement syntax and FP-like expression syntax

I quite like this though. All other expression based languages like MLs and Haskell use indentation, making it very hard to parse. To me, Rust's syntax allows very granular control. I wish more things in the language can become expressions, like the let expression, for/while loop.

Re:Two Years with Rust by dgb23 on Tuesday March 24, 2020 @01:58PM (#22676934)
I agree that it is useful. But from the perspective of a learner it is a stumbling block. Rusts syntax seems to be a compromise to cater to c-likeness. Not a deal breaker at all, just something that I think is not as good as it could have been and will confuse adopters at least in the beginning.

On the flip-side it might attract more developers who are not familiar with anything that is not c-like.

As a side note, Lisps are expression based and have a very clean syntax where you don’t need to mentally construct an AST, it is literally there in front of you already.

Re:Two Years with Rust by gameswithgo on Monday March 23, 2020 @09:11AM (#22663752)
It may take a long time to get comfortable with how to program in a way that satisfies the borrow checker. How much effort this takes will depend on your brain, and the type of projects you tend to work on. There are a lot of programs you can make where you just don't tend to run into borrow checker issues and everything is easy! Others it can immediately be a huge pain, and very frustrating.

It took me a few attempts at Rust to start building good intuition for it.

Re:Two Years with Rust by steveklabnik on Monday March 23, 2020 @09:00AM (#22663659)
> Are compilation speeds an issue for anyone?

It is still our #1 complaint, and something we're still working on.

I recently came across a crate that still maintains compatibility for Rust 1.13. It compiles 2.75x faster on 1.41.1. We've made a lot of progress but there's still a lot of more work to do.

Re:Two Years with Rust by modernerd on Monday March 23, 2020 @03:41PM (#22667907)
Thanks so much for the reply, and for your efforts! So great that this is still being actively tweaked.
Re:Two Years with Rust by steveklabnik on Monday March 23, 2020 @03:55PM (#22668062)
You're welcome for the reply. The efforts are not mine so I can't take credit for them :)
Re:Two Years with Rust by the8472 on Monday March 23, 2020 @08:16AM (#22663268)
> Are compilation speeds an issue for anyone?

It really depends on what you're doing. An incremental change on a small project can debug-build in seconds on a beefy machine, syntax-check is even faster. If you're working on rust itself and do a full 2-stage rebuild to benchmark something and upstream changed the llvm revision in the meantime and you do that on a laptop it can take more than an hour.

Re:Two Years with Rust by atilaneves on Monday March 23, 2020 @08:51AM (#22663562)
> Are there any downsides beyond the niggle the author mentioned?

For me, using a GC is just a lot easier than making the borrow checker happy. Given that in practice it's rare that the GC is the bottleneck (and when it is I write around it after profiling), the trade-off just isn't worth it for me.

> Are compilation speeds an issue for anyone?

Yes, it's pretty slow to compile.

Re:Two Years with Rust by imtringued on Monday March 23, 2020 @02:16PM (#22666935)
You can't "write around" the GC in Java. The JVM will reach the max heap size quickly unless you avoid all allocations.
Re:Two Years with Rust by pjmlp on Monday March 23, 2020 @05:29PM (#22668989)
There are plenty of GC enabled systems programming languages that offer all C++ set of features for memory allocations, besides the GC.

Java isn't one of them, at least until Valhalla comes along.

Still there are Java APIs for off GC heap allocation, which bypass that limitation.

Re:Two Years with Rust by benschulz on Monday March 23, 2020 @09:01AM (#22663670)
The biggest downside for me is that from time to time (rarely) the compiler really lets you down.

Right now I'm debugging the equivalent of #64650[1] but the cause appears to be distant from where the error is reported, triggered by monomorphization. I've seen other compiler behaviors in the context of async code that the maintainers could not explain.

That said: The compiler is a marvel, otherwise. Never have I had as much help from a compiler. I guess that's because most other languages have much less complex problems to communicate.

[1]: https://github.com/rust-lang/rust/issues/64650

Edit: Another data point is https://github.com/rust-lang/rust/issues/60658

Re:Two Years with Rust by _bxg1 on Tuesday March 24, 2020 @12:06AM (#22671791)
Compilation speeds are perfectly fast for small projects, especially without compiler optimizations, especially when re-compiling (it only re-compiles what changed). I certainly wouldn't call that one of the barriers to trying it out.
Re:Two Years with Rust by winrid on Monday March 23, 2020 @12:41PM (#22665742)
The downside is that the performance payoff is really small compared to easy to write languages like Java for most projects, and you will write your code slower.

Compilation is slow.

But your code will be reliable.