News for hackers. Stuff that matters.
Re:Ryzen Threadripper Pro 3995WX Spotted by baybal2 on Friday July 10, 2020 @07:49AM (#23790549)
> Building software

Well, the problem with that is that no make system can utilise so much core. Even linux kernel, probably the biggest C project in mainstream use, can not consistently load even 16 cores with mostly handwritten makefiles.

P.S.

I do not say that building software is not CPU parallelisable in principles, I'm saying that regular make systems have trouble handing big number of parallel tasks in practice, even in big, well run projects that use handwritten makefiles.

Re:Ryzen Threadripper Pro 3995WX Spotted by ajross on Friday July 10, 2020 @07:52AM (#23790572)
That's... not true at all. A kernel build will be as parallelizable as you like. I do kernel builds fairly regularly on 56 and 64 core hosts and in fact the build time is quite linear with core count. (The ramdisk packing provided by the distro at "make install" time tends to be what hurts scaling, FWIW).

And my day job involves lots of iterations over the Zephyr test suite, which builds and runs hundreds of individual test apps X dozens of platforms. This is almost a pure build-time problem and scales very linearly indeed.

Re:Ryzen Threadripper Pro 3995WX Spotted by CrazyStat on Friday July 10, 2020 @08:56AM (#23790990)
CPU-parallelizable and GPU-parallelizable are different problems. CPU cores are independent. GPU cores are SIMD--every core does the same operation, but with different data. That works really well for things like matrix multiplication where you need to multiply a million times and then add a million times, because you can have all 3000 CUDA cores multiplying and then all 3000 CUDA cores adding. It doesn't work for compiling software.
Re:Ryzen Threadripper Pro 3995WX Spotted by monocasa on Friday July 10, 2020 @11:02AM (#23792174)
It's not that SIMD for GPUs. It'll be 64 or 32 GPU threads together generally sharing the dispatch and control logic. So if you have a 4096 "CUDA core" GPU, it'll be 64 of what would be called cores on the CPU side, each with a 64 lane vector processor.
Re:Ryzen Threadripper Pro 3995WX Spotted by baybal2 on Friday July 10, 2020 @08:54AM (#23790971)
Not in my experience, from time to time it does boost to 40+ compile tasks, especially on drivers code, but it mostly hover at around 10 tasks.
Re:Ryzen Threadripper Pro 3995WX Spotted by emidln on Friday July 10, 2020 @07:56AM (#23790595)
Bazel utilizes the cores near linearly. It even makes use of remote cores in an efficient manner for compiling c++. 128 thread and 256 thread machines are great if you also have enough ddr4 to let each gcc/clang do their thing.
Re:Ryzen Threadripper Pro 3995WX Spotted by pdimitar on Saturday July 11, 2020 @09:41AM (#23802133)
The rumoured Threadripper Pro 3995WC (64c/128t but with 8-channel memory support) might solve that problem.
Re:Ryzen Threadripper Pro 3995WX Spotted by jchw on Friday July 10, 2020 @09:15PM (#23798515)
Something is wrong on your end, you can parallelize the build of typical C and C++ software almost infinitely. The reason why is because most C and C++ software do not have 'happens-before' dependency relationships due to the way translation units work, so basically every translation unit in a module can be compiled 100% independently of any other translation unit. In practice that means you can easily scale up to 1000+ processes with no issue, and in fact, this has been accomplished in combination with AWS Lambda:

https://stanford.edu/~sadjad/gg-paper.pdf

Actual screencast of it in action compiling FFMPEG:

https://asciinema.org/a/257545

(Perhaps the issue is you are bottlenecking on I/O or some other resource?)