News for hackers. Stuff that matters.
Re:The Rust compiler isn't slow; we are by imtringued on Wednesday June 17, 2020 @05:15AM (#23549400)
I seriously hope the Rust community won't get used to these slow compile times the way the C++ community got used to it. C++ has some really big flaws that can cause compilation time to become unbearable but for some reason boost really loves to abuse these flawed features. Boost is almost as bad as npm. You include a single header and then you end up including thousands of other boost headers. When I looked at the include hierarchy of a single cpp file 99% of the includes were from boost and maybe 30 includes were actually from the C++ project itself. I'm sure you could cut down 80% of the compilation time simply by removing boost and replacing it with different libraries.

I even found a reddit submission that shows how absurd boost is: https://www.reddit.com/r/cpp_questions/comments/2hzobl/reduc...

Re:The Rust compiler isn't slow; we are by est31 on Wednesday June 17, 2020 @08:22AM (#23550493)
On the bright side, the compiler's performance is improving constantly. Timing the compilation of lewton plus its dependencies on the 0.9.4 release commit:

* 8.042s on rustc 1.20.0 from 2017-08-27

* 5.061s on rustc 1.30.0 from 2018-10-24

* 4.789s on rustc 1.36.0 from 2019-07-03

* 3.197s on rustc 1.44.0 from 2020-06-01

Re:The Rust compiler isn't slow; we are by fluffything on Wednesday June 17, 2020 @06:05AM (#23549699)
In my experience, Rust compiles orders of magnitude faster than similar C++ code for initial compiles. For incremental compiles, Rust is infinitely faster. C++, even if you use Modules, needs to re-compile, re-instantiate, re-check all the templates all the time. I speculate that this alone is what probably is responsible for the largest difference in compile-times.

Comparing Rust with C here is hard. As others have mentioned, with C, you typically only use libraries that are installed in your system. Rust does not have a global system cache for libraries, and I don't think there is any package manager installing pre-compiled Rust libraries, so at least for the initial compiles, Rust needs to do much more work than C, for Rust dependencies at least. If your Rust project only depends on C libraries, then there is no work to be done of course.

For incremental compiles with C, it is hit or miss. If your C project is properly "modularized" (split into a lot of small enough TUs), then when you change a TU, C only needs to recompile that one TU. In our benchmarks, Rust is faster for that situation, and speculating, this is because the Rust compiler only recompiles the parts of a TU that actually changed.

In practice, however, Rust is much slower than C, because people do not write small Rust TUs, but absurdly huge ones, at least, by C standards. A Rust TU (crate) usually is mapped 1:1 to a C library, which often contains dozens or hundreds of TUs. Just the cost of testing what changed in a Rust crate is often larger than the cost of blindly recompiling the C TU that has a newer timestamp.

If people would split Rust code like they do with C, then this wouldn't happen. But Rust people are too optimistic in that one day the compiler will be able to somehow auto-magically split a crate into multiple TUs as good or even better than what people do for C manually. And do that determination + the incremental compile faster than what takes a C compiler to blindly recompile a tiny amount of code.

Re:The Rust compiler isn't slow; we are by jfkebwjsbx on Wednesday June 17, 2020 @08:26AM (#23550531)
> Rust compiles orders of magnitude faster than similar C++ code for initial compiles

Please avoid misleading statements like "orders of magnitude faster". If that were true, even super-complex projects would compile in a few seconds, given it would be at least 100x faster than compiling C++.

> For incremental compiles, Rust is infinitely faster.

Please avoid obvious exaggerations...

Incremental compiles of C++ take as low as modular is the code. Same for Rust. If you make huge TUs, you get huge incremental compile times.

> C++, even if you use Modules

Modules are not even used yet by any big project nor properly supported yet, and they are not expected to speed up compilation much anyway.

> Comparing Rust with C here is hard.

It isn't hard: C can be compiled way faster than either C++ or Rust because the code and type system is much simpler.

> As others have mentioned, with C, you typically only use libraries that are installed in your system

C code can be anything. C++ libraries are also included with the system.

> Rust does not have a global system cache for libraries

It does have a cache per user/project/etc., which is basically the same and without it the speed would be atrocious.

Re:The Rust compiler isn't slow; we are by fluffything on Friday June 19, 2020 @05:53PM (#23579229)
> Please avoid misleading statements like "orders of magnitude faster". If that were true, even super-complex projects would compile in a few seconds, given it would be at least 100x faster than compiling C++.

The tests of my Rust range-v3 like library compile in ~12 seconds. The tests of C++'s range-v3 take ~320 seconds to compile.

> Please avoid obvious exaggerations...

The incremental compile of the same library takes in Rust ~1 second. In C++, it still takes ~320 seconds, zero improvement because C++ header-only generic libraries do not benefit from incremental compilation.

> Modules are not even used yet by any big project nor properly supported yet, and they are not expected to speed up compilation much anyway.

I maintain a ~500kLOC C++ project that uses modules. C++ Modules did not speed up compilation at all over simple PCHs.

> It does have a cache per user/project/etc., which is basically the same and without it the speed would be atrocious.

No it does not. Check out any project from crates.io into a subdirectory, do a cargo build. Check it out into another sub-director, do a cargo build. All dependencies are re-built.

You'd need to go way out of your way and set up like sscache to get something similar, but Rust and cargo do not do this.

> It isn't hard: C can be compiled way faster than either C++ or Rust because the code and type system is much simpler.

Citation needed. On my machine, rustc spends ~80% of its time in LLVM, which is slow, because the translation units are huge. The other 20% is split into ~5% parsin, ~5% type checking, and ~10% LLVM-IR generation.

Even rustc's synthetic trait metaprogramming benchmarks do not spend even 10% of the compile-time in type checking.

So if you have a test for which rustc spends most of its compile-time in type checking, go ahead, and post a link. We'd love to add it to rustc's benchmark suite.

Also, on my machine, while clang spends less time in code generation, it is still over ~60%.

Clang spends more time in parsing and type checking, and that's because C++ overload resolution scales with the number of visible overloads, and the more header files you have, the more overloads there are (e.g. an error with operator<< on a ~500kLOC project produces ~2000 "potential candidates"). While that's part of the slowdown, still 60% of the time is spent in LLVM. The reason is that idiomatic C++ just produces more code than C due to templates, inlining, etc. for similarly-sized TUs.

Parallelizing compilation within a TU is hard to do well. Rustc tries without much success and clang does not. In C, you can simply parallelize by using more compiler processes to compile fully independent TUs in parallel. These TUs are also smaller because idiomatic C code doesn't use many macros to generate code, most functions are linked opaquely, generics also generate few code via void*, and the worst culprit of long compilation times in C, which is including too many header files, can properly be cached using PCHs.

Re:The Rust compiler isn't slow; we are by Kaze404 on Wednesday June 17, 2020 @09:32AM (#23551141)
This type of comment is the worst. What did you add to the conversation besides a bunch of useless "well ackshually"? If you want to respond to a comment, respond to it in full, not only conveniently out of context sentences.
Re:The Rust compiler isn't slow; we are by mlthoughts2018 on Wednesday June 17, 2020 @11:09AM (#23552341)
The comment makes good points that need to be considered and helps mitigate the unrealistic exaggerations of the parent comment.

This does not fit the “well akshually” trope at all.

Re:The Rust compiler isn't slow; we are by Nullabillity on Wednesday June 17, 2020 @06:51AM (#23549967)
> Rust does not have a global system cache for libraries, and I don't think there is any package manager installing pre-compiled Rust libraries, so at least for the initial compiles, Rust needs to do much more work than C, for Rust dependencies at least.

FWIW, Nix can do this (via Carnix). But it turns out not to be too useful anyway, because any feature flags that you use will invalidate the cache for everything that depends (transitively) on that crate.

Re:The Rust compiler isn't slow; we are by Mic92 on Wednesday June 17, 2020 @08:09AM (#23550404)
crate2nix is better maintained alternative to carnix: https://github.com/kolloch/crate2nix The cache is still useful though if you think about CI builds or having a common project in a team.
Re:The Rust compiler isn't slow; we are by quietbritishjim on Wednesday June 17, 2020 @06:12AM (#23549741)
For C++, if you have many library headers used repeatedly in many source files, the number 1 suggestion ought to use a precompiled header (which you write yourself with commonly used library headers but none of your application headers). It was mentioned on that page but a bit buried and OP replaced a whole dependency before trying it.

I don't think the same thing would work for Rust because, unlike C++, it doesn't need to constantly reparse files in its depedencies - essentially it already has them automatically. C++ is aiming to move that way in future with modules.

Edit: I just tried turning off precompiled header on one of our larger programs that's heavy on Boost and a couple of our other libraries, and it increased build time from 2:16 to 10:52 (that's minutes and seconds) after fixing a few missing #includes that were covered by the precompiled header.

Re:The Rust compiler isn't slow; we are by usefulcat on Wednesday June 17, 2020 @09:34AM (#23551169)
It seems to depend a lot on the particular compiler. I know that in the past, precompiled headers made a huge difference with MSVC (and maybe they still do, but I haven't used it in some years at this point).

Today, I have a medium-sized c++ project that uses Linux, gcc 9.2 and boost and I have yet to find any combination of headers that I can put in the precompiled header file and see any reduction in compile times, either for a full build or an incremental build.

Re:The Rust compiler isn't slow; we are by quietbritishjim on Wednesday June 17, 2020 @05:47PM (#23557177)
Hmm, that is interesting... You're absolutely right, the program I was compiling is a Visual Studio program. (The code is actually cross-platform, partly thanks to the use of Boost, but for historical reasons it only has a Windows build system in the form of Visual Studio projects.) I'd definitely like to try my experiment again with GCC or clang at some point.
Re:The Rust compiler isn't slow; we are by silon42 on Wednesday June 17, 2020 @06:25AM (#23549814)
Precompiled headers are a workaround when you can't modularize your project / headers files. But to use them you commit to the wrong way, including everything the same way everywhere.... what you should do is include only minimal stuff from the headers, and prefer to include most dependencies only in implementation.
Re:The Rust compiler isn't slow; we are by quietbritishjim on Wednesday June 17, 2020 @06:52AM (#23549973)
> Precompiled headers are a workaround when you can't modularize your project / headers files.

This is technically true but misleading - the whole nature of C++ means that you can't modularize your project / headers in a way that avoids transitively including lots of headers. There's two reasons for this:

* Lots of Boost (and other library) types are vocabulary types - that's often the whole point of using them in the first place. If my class has a public method that returns a boost::optional<T> or boost::flat_map<K, V> then there's no way to avoid transitively including the relevant Boost headers in my header files.

* Lots of Boost stuff is templated so the implementation has to go in the header, which means the transitive dependencies on other parts of Boost have to be included in the header, even they're only used in the implementation of the Boost class that you're directly use. The heavy templating is a result of the previous point - vocabulary types often need to be templated in order to be useful in many different situations.

Re:The Rust compiler isn't slow; we are by Rexxar on Wednesday June 17, 2020 @12:11PM (#23553062)
You just have to use forward declaration headers in you headers. For your two examples :

-> flat_map: https://www.boost.org/doc/libs/1_67_0/boost/container/contai...

-> optional: https://www.boost.org/doc/libs/1_67_0/boost/optional/optiona...

Re:The Rust compiler isn't slow; we are by quietbritishjim on Wednesday June 17, 2020 @05:45PM (#23557146)
That's true, but it creates an annoying extra burden on anyone that calls those functions to include the full-fat library header themselves. Since they might simply write `auto x = my_obj.returns_optional()`, it might not even be that obvious they need to add that header.

Plus, of course, once they have added that header to their file, that header is included in their file, if you see what I mean, so it may not actually save compilation time while still adding coding effort. Really, I think that technique is only useful if you're only referring to those types (a) in private functions, and (b) by reference or pointer. By the time it's a public function, you have a responsibility to include the full library header in your header file.

Re:The Rust compiler isn't slow; we are by pjmlp on Wednesday June 17, 2020 @07:16AM (#23550100)
Which is why many companies shy away from Boost and from libraries that do clever template meta-programming.
Re:The Rust compiler isn't slow; we are by quietbritishjim on Wednesday June 17, 2020 @08:05AM (#23550386)
Indeed, but it's just ignorance driving that. You end up with three choices:

* Use libraries like Boost and waste developer time waiting for compilation.

* Avoid libraries like Boost and waste developer time re-implementing functionality (but then you have the same problem anyway) or suffering from not having that functionality available to them.

* Use libraries like Boost and spend a small one-time developer cost turning on precompiled header files in your build system.

Re:The Rust compiler isn't slow; we are by mmm_grayons on Wednesday June 17, 2020 @11:20AM (#23552486)
Boost is a pain to compile, sure, but a bit of a necessary evil for anything C++. Threadrippers are dirt cheap compared to developer time and compilation is embarrassingly parallel. Just by devs 32-core boxes and be done with it.
Re:The Rust compiler isn't slow; we are by pjmlp on Wednesday June 17, 2020 @12:18PM (#23553164)
Except that on the projects I have been consulting, the last time I actually coded on a desktop was around 2006. And most customers nowadays just have IT throw a VM to externals.

You're not getting Threadrippers from these kind of companies.

Re:The Rust compiler isn't slow; we are by MaxBarraclough on Wednesday June 17, 2020 @06:44AM (#23549925)
So things would be better if Boost.Signals2 weren't header-only? Makes sense.
Re:The Rust compiler isn't slow; we are by amelius on Wednesday June 17, 2020 @07:55AM (#23550331)
Compile time is not the only reason why Boost is absurd. Have you looked at the code? :)