Past a certain point, long compile times make it harder to improve a large code base. People work around it by adding more stuff at the leaves and never fixing fundamental parts of the code base, avoiding to touch heavily shared code. This is almost a guarantee for another increase in compile time (because you end up accreting more stuff) which makes it even harder to work on the code base at large scale.
And that has I believe a concrete effect on the quality of a piece of software.
The problem is, there are some low hanging fruits (like caching, parallelizing and distributing builds) so people think they can delay the inevitable. The problem is, those counter measures only deal with the average case, while doing nothing about the worst case.
Having looked at the clang modules implementation you basically have to clean up all the system includes, the standard library and your own headers before you can use them. And you will have to rewrite many things in these headers due to changed preprocessor semantics.
I think neither the clang nor MSVC implementations can be used for anything but the most trivial code due to these issues.
There are talks from Google who invested heavily into modules using clang and got gains, but as far as I understand they fixed those issues only for themselves.
More info welcome.
At this point the only cure is brute force (blades with many cores are not that expensive, or a distributed compile cluster) and always taking care with your includes.
Aren't modules especially limited with the mentioned problem of templates? If I have a vector of a custom type, I still have to instantiate it in every file that is using this type, I don't think modules help much here, they reduce the amount of time parsing, not instantiating if I understand correctly.
We'll need to significantly change the way we handle templates in compilers to get significant improvement.
I thought Google fixed the C++ compile time problem by inventing Go?
Go is hardly a replacement for C++. Garbage collection, no templates, no const, etc.
I am confident that by C++20 they will be available.
The main problem will be the adoption by major libraries.
Regarding the libc and other headers, modules will still fullly support include files, so that won't be an issue. You should be able to include headers and export them as modules.
libc and libstdc++ are part of the compiler runtime, ie. will be updated together with it.
Kernel headers are only an issue on free UNIX like OSes, on other OS what matters is what the SDK provides.
So at least in what concerns Apple, Google, Microsoft and every other commercial vendor OSes, don't expect a big issue having their SDKs updated when they start supporting modules.
The problem is getting the BSD, Linux and open source libraries without a specific agenda to adopt them.
I found a really simple solution to this: disable all optimizations while you are developing and only use them for production build. I have 70k LOC project that compiles from scratch in 10-11 seconds (using 8 parallel builds).
Regular changes to 2-3 .cpp files and link are done in 1-2 seconds.
This is esp. important with MSVC which has a huge difference between non-optimized and optimized build.
Also, use precompiled headers.
If you listen to the C++ advocates, they all say use the STL as much as possible, use boost if there is solution for your problem there, etc.. It's true, you probably will quickly get a solution that works and looks pretty, but if you do this a hundred or a thousand times your code suddenly needs several hours to compile.
I am not against the STL, or boost, but there is a cost associated with using them that is often ignored, but IMHO is actually a productivity killer in codebases that are a bit bigger.
While Boost often implements the latest proposed standards and provides a vast extension to the STL, it is not the STL. Often proposed standards changes are best implemented by changing the compiler itself. Expecting Boost to provide the same functionality as a modified compiler without a cost is foolish.
Boost provides great implementations but at a great cost in complexity, compile time and binary size bloat. I don't even really believe modules will solve this. They push the boundaries of what is possible with today's equipment, in order to discover what the next equipment should be.
Also I no longer look to Boost libraries as a Seal of Quality. There be dragons. Choose wisely.
For high precision currency, use int64_t (perhaps in a simple POD class). This, with an 8-digit implied decimal part, is enough to represent the finest price unit in any market today, with a max of 92 billion.
If you are dealing with more than 92 billion of any currency in a single place, don't complain!
"Bank accounts with balances of up to 175 quadrillion Zimbabwean dollars will be paid $5. Those with balances above 175 quadrillion dollars will be paid at an exchange rate of $1 for 35 quadrillion Zimbabwean dollars.
"The highest – and last – banknote to be printed by the bank in 2008 was 100 trillion Zimbabwean dollars. It was not enough to ride a public bus to work for a week."
https://www.theguardian.com/world/2015/jun/12/zimbabwe-offer...
Gas stations in the US had a similar problem in the 1970s; most pumps didn't go over 99 cents a gallon. Took a few years for things to be retrofitted.
I'm sure no one dealing with zimbabwean currency would care for so many significant digits.
But you certainly could have a fast path for stable currencies that are very unlikely to hyperinflate (USD, EUR, GBP, JPY).
Even in EUR/USD/GBP, a typical large bank balance sheet is over a trillon.
In my particular case, 64 bit integers might be sufficient. I thought about it but then decided I wanted to be on the safe side. Maybe not the best decision.
So I wanted perform the caculations with high precision, then round the final result according to what the law requires.
How so? If there's a resting offer at 11.58726 and I put in a buy order, limit 11.63492, there's no reason an exchange can't match that and execute either at the midpoint or at 11.58726 +/- maker/taker (whichever the exchange policy is). What's the problem?
My understanding is that all prices on exchanges have a fixed number of decimal (and sometimes are even based on less granular base, like 1/8th of a decimal).
There's an argument that more granular prices make it harder for humans to read the order book but I don't put much faith in it.
http://clang.llvm.org/docs/Modules.html#problems-with-the-cu...
Though I'm curious as to what you mean by "unbearably slow", and what your toolchain is.
I find the IDE to still be sufficiently fast after disabling some things like Git integration and codelens.
In fact, a lot of modern software engineering guidelines come from working around C++'s built in shortcomings.
Use things such as the Pimpl idiom to help linking. Modular libraries rather than big-ball-of-mud, do you really need to recompile world+dog each time? Top-down includes, and no relative includes, isolating your code. In fact go-lang does exactly this.
The trouble is that in C++, fast compile times need to be designed into the code base and often enforced by scripts. I don't think that there's any other language that requires so much 'non-language' engineering as C++.
what is unbearably slow ? an hour ? a day ? how many times did you actually have to rebuild the entire code base ? Did you do parallel builds, or see what gains you could get from faster hardware ?
I'm genuinely curious as I've never worked on a codebase that large, and the only time you have to rebuild everything is when someone changes a common library header.
I'm not paid to use them or shill for them I just think distributed compilation might help
Can you put numbers on these figures? By "Unbearably slow" how many minutes (or hours) did you mean? Do you use a heavy desktop with a fast SSD and a lot of RAM?
Finally what is the total output program size (roughly). Like are we talking about an application that is in the gigabytes, like the complete Adobe suite or something?
I would like to know more information about how it's possible for this to happen. tell us as much as possible :)
Thanks!
Another thing is to rely on the preprocessor for code generation; then you're dealing with memory-speed elaboration of the AST, rather than reading off of disk.
In C and "older" C++, you're probably used to doing a particular style of separate compilation. You put your prototypes in header files, your implementations in cpp files, do the equivalent of `gcc -o somefile.o somefile.cpp` for every cpp file in your project. Each of those separate compilation stages has to include any referenced headers, but then it just compiles the code in that cpp file and writes an object file. At the end, you do one more step to link all the object files into a binary executable. This is fast and as parallelizable as the number of files you have.
In C++ today, you're likely doing a lot with templates, and templates don't work this way. With templates, every compilation unit that references a template needs not just declarations from a header file -- it needs the complete source code for that template. You essentially have to #include the cpp files (through one mechanism or another).
Simplifying a bit, that means that if you have a template defined in one file that is used by code in 25 other files, that template gets compiled 25 different times, and worse, you lose out on a lot of parallelism because by the time the compiler sees the code, the preprocessor has expanded the templates into one giant compilation unit that's going to be (slowly) compiled on one core.
I just grabbed the code I wrote during my PhD thesis that was heavily template-based. It's tiny (about 15,000 lines of C++) and building it on this $3000 2015 Macbook Pro took about a minute, and clang++ used a bit over 800 MB of memory in the process. (https://github.com/deong/sls if anyone cares to see the example).
You certainly don't have to be building the Adobe suite to see compile times in the tens of minutes, and even moderate sized applications can put you into the hours. So much just depends on how the application is structured and written as opposed to only looking at size.
"Unbearably slow" is when the compiler takes too long for me to stay focused on a single task. So compile times in the minutes are a blow to my productivity.
The machine is i7, 4 cores, 8G RAM, SSD. Definitely not as beefy as it should be.
But my point isn't about how to optimize C++ compile times by adding more hardware. It's about the relative impact of "modern" C++ compared to traditional C++. There shouldn't be a disincentive to using modern libraries.
¹ I used it in a couple projects and never had a single incorrect build. With most other build tools, be it CMake or Gradle, these happen regularly for me.
All the new stuff is heavily template based. As more libraries are written in that style, compile times go through the roof.
I wrote some code recently that required high precision monetary calculations. So I used boost multiprecision. It turned out to be completely impossible as most compilation units ended up depending on that library directly or indirectly.
Already slow compiles became unbearably slow. Facing a tight deadline I quickly wrote an extremely ugly, slow, error prone, pimpl based wrapper around boost multiprecision. Many calculations now require dynamic memory allocation for no good reason.
This is just terrible, because I know I can't leave it like this. I'm going to have redesign the whole thing or look for other libraries or look a lot more deeply into the calculations to see if they really cannot be done with large integers.
It's a productivity disaster. You could say I made wrong choices, which is true, but I think the C++ standards committee has been making wrong choices in not prioritizing a fix for compile times (modules?).
What's the point of having nice new features if you end up spending half your time on carefully cordoning off any code that uses them?