News for hackers. Stuff that matters.
Re:Understanding Rust futures by going way too deep by kungito on Monday July 26, 2021 @05:30AM (#27957404)
I'd say it's pretty unfair to take clean build time to consider doing something useful. 98% of my rust builds are incremental builds which take 10-15 seconds for a project I have been working on for 9 months full time and I think my dependency tree has easily 1000 dependencies.
Re:Understanding Rust futures by going way too deep by DixieDev on Monday July 26, 2021 @05:48AM (#27957507)
True, it's not a real problem in most scenarios if incremental compile times are good. I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share.

Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause, and possibly multiple pull requests to get it fixed. I think my CPU would spend a bit longer than 10-15 secs on the linker step, too.

Re:Understanding Rust futures by going way too deep by fasterthanlime on Monday July 26, 2021 @06:49AM (#27957909)
Compile/link times are definitely a discussion topic. Here's some quick tips:

  * Incremental compilation helps locally, not so much in CI, unless you save/restore the whole cache which gets large quickly and evicting the out-of-date objects is non-trivial.
  * In CI, sccache helps, but not as much as I'd like: there's a bunch of things that are non-cacheable, notably crates that pull in & compile C/C++ code (I'd trade 2 c-bindings crates against 12 Rust-only crates any day for that reason alone)
  * Splitting stuff across different crates helps parallelizing the build (and caching it better), which may be one of the reasons some projects end up having "1000 dependencies" (although I've rarely seen upwards of 600)
  * For incremental builds, linking does become the bottleneck. Full LTO is especially slow, Thin LTO is better. Switching to LLD improves thing. I'm hopeful that mold will improve things some more.
  * Re multiple PRs: thankfully crate families tend to live in the same repository, so fixing something across both tracing / tracing-subscriber could be a single PR, for example.
I wish the Rust community invested more in build caching, but even with the current state of things, there's often steps you can take to make things better.
Re:Understanding Rust futures by going way too deep by snovv_crash on Tuesday July 27, 2021 @04:24AM (#27969963)
Can you combine ccache with a sccache to make the C/C++ parts fast too?
Re:Understanding Rust futures by going way too deep by styluss on Monday July 26, 2021 @10:59AM (#27960213)
Have you tried sccache? https://github.com/mozilla/sccache
Re:Understanding Rust futures by going way too deep by riquito on Monday July 26, 2021 @11:25AM (#27960588)
It's his second bullet point
Re:Understanding Rust futures by going way too deep by volta83 on Monday July 26, 2021 @08:03AM (#27958397)
I rather have 1000 dependencies than re-implement all those 1000 dependencies myself, chase all the bugs that have already been ironed out in those, etc.

My time is worth more than my computer's time.

Re:Understanding Rust futures by going way too deep by myrrlyn on Monday July 26, 2021 @07:17AM (#27958066)
the reason for

> I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause

is that there is far, far more likely to be a bug in the version you write on your own to achieve the same goal than there is in a widely-observed library written by somebody who's chosen to specialize in that specific thing

Re:Understanding Rust futures by going way too deep by hnlmorg on Monday July 26, 2021 @08:00AM (#27958370)
In theory yes. But in practice that isn't always true. People often don't audit other modules on the assumption someone else had. Which means nobody ends up doing it. And if you end up with an ecosystem that favours more modules over fewer, you can end up with more modules than a given developer or team are willing to audit (a bit like "alarm fatigue" where if you have too many objects to check then people will inevitably just get lazy).

Just look at how many C and C++ libraries are maintained by 1 individual and have almost no 3rd party oversight to see that Rust can't automatically make the claim you made.

That all said, for anything complicated and/or directly security related, one should always check if there is a module first.

Re:Understanding Rust futures by going way too deep by db48x on Monday July 26, 2021 @09:33AM (#27959136)
I look at it the other way around. You own any bug in your product whether it comes from a dependency or from code of your own; you have to fix the bug either way. Using a dependency doesn’t reduce your responsibility, but it does reduce the amount of code that you have to write yourself.
Re:Understanding Rust futures by going way too deep by hnlmorg on Monday July 26, 2021 @10:14AM (#27959590)
But if you are willing to own that responsibility then you should read the code you're importing to begin with. I know I do but I also know most people don't bother.

I do acknowledge that there will always be bugs that are identified by your users but equally if you're not auditing your dependencies first then it's hard to argue that you're not just passing off that responsibility wholesale to your users.

Re:Understanding Rust futures by going way too deep by kungito on Monday July 26, 2021 @11:46AM (#27960881)
It's always a tradeoff whether you want to read some other code or work on something else. Rust ecosystem is not that mature so for a few libraries I had to end up rewriting the thing myself with some fixes or without some bloat. I'm writing an application level thing and I need as many utilities as possible as I do not want to write all the layers for all the abstractions that end up in my product. Then when something breaks I investigate, offer a fix, open an issue or whatever. I'm not writing something that requires too much reliability or whatever, the utility is elsewhere.
Re:Understanding Rust futures by going way too deep by usefulcat on Monday July 26, 2021 @08:35AM (#27958608)
> there is far, far more likely to be a bug in the version you write on your own

In general I agree with this, but there is another relevant aspect to consider: something that I've written on my own for a particular project is also likely to be more purpose-built, and therefore simpler.

Re:Understanding Rust futures by going way too deep by kelnos on Saturday July 31, 2021 @04:31AM (#28016442)
> Having 1000 dependencies sounds crazy to me!

I was thinking that too, but then I did a quick check of a few of my Java services for work, and many of them have more than 400 dependencies (most transitive, of course), with a few getting up to 600.

Obviously comparing Java and Rust is not exactly apples to apples, but I rarely care much about the number of dependencies in my Java projects, so I guess I shouldn't be too worried about Rust projects either (aside from link time, I guess, which javac doesn't have to do).