News for hackers. Stuff that matters.
Re:Base64 encoding and decoding at almost the speed of a mem... by m0zg on Wednesday November 06, 2019 @02:32AM (#21460460)
Many people don't realize this but today's memory is dramatically underpowered for today's CPU. Consider the following: you can, theoretically, get ~80GB/sec of memory bandwidth from a modern Intel CPU (assuming you use all channels, and there are no interrupts, etc). That's 20 billion floats or int32s per second. The same CPU can do ~2.3 fp32 TFLOPS. That's 115 ops for every fp32. And it's getting worse as more and more cores are put on the same memory bus.
Re:Base64 encoding and decoding at almost the speed of a mem... by Const-me on Wednesday November 06, 2019 @04:07AM (#21460844)
Your numbers look strange. Did you quoted from a $10000 xeon platinum very few people use?

In my current desktop PC, it’s 700 GFlops theoretical compute versus 50GB/sec of theoretical RAM bandwidth.

Re:Base64 encoding and decoding at almost the speed of a mem... by m0zg on Wednesday November 06, 2019 @04:40AM (#21461009)
For i9-7980XE Intel quotes 1.3TFLops of fp64. So it ought to do about twice that at fp32. Yours at $1500+tax from Newegg: https://www.newegg.com/core-i9-x-series-extreme-edition-inte.... Quad channel RAM, too.

For a flea check, consider that the chip has 18 cores and runs at ~4.2GHz. To achieve the theoretical throughput each core has to do ~30 single precision flops per cycle. Which sounds about right: HEDT intel chips can (theoretically) do 32 of those per cycle.

This, of course, ignores whatever throttling Intel would need to apply to maintain TDP, so a purely theoretical back of the envelope.

Re:Base64 encoding and decoding at almost the speed of a mem... by satanspastaroll on Wednesday November 06, 2019 @03:43AM (#21460723)
This should change soon, as EPYCs can do 204GB/s of memory throughput (plus tons of pcie4.0). They also aren't cpu tied, all of the SKUs get all of the lanes
Re:Base64 encoding and decoding at almost the speed of a mem... by m0zg on Wednesday November 06, 2019 @04:58AM (#21461102)
It's a NUMA though. You'll have to write software specifically for NUMA to get anywhere close to that number. To make matters worse, EPYC also doubles the number of cores. To be fair, though, it also has a substantial amount of cache, but cache is not a panacea if you need something in the main RAM. And if you're churning through gigabytes of stuff per second, you'll be needing that very, very often.
Re:Base64 encoding and decoding at almost the speed of a mem... by gmueckl on Wednesday November 06, 2019 @09:43AM (#21463026)
Yes, NUMA and large numbers of core will pose completely new challenges for software optimization. Applications will have to become aware of the memory architecture of the underlying machine. They will have to make explicit assignments of memory allocations and threads to NUMA nodes based on their domain specific needs. In some cases, even duplicating data structures may be the right call. This is going to challenge how most developers write fast software.

The other thing is that even seemingly trivial things like spawning and synchronizing with lots of threads will be much more complex on CPUs with many cores. At some point, naively looping over all threads is going to be too slow. I think that the limit is going to be around 64 cores. Past that, you should actually parallelize your worker thread management to stay efficient. There is precendent for this in HPC, e.g. MPI implementations.