But what I can say, is that AVX and AVX2 are rather limiting. And I want those new instructions that are found only in AVX512.
Not even to run 512-bit computations mind you. I want to run 128-bit computations in AVX512. You can use XMM registers in AVX512 ya know!!
The main benefits of AVX512 are the mask registers, scatter-instructions (AVX / AVX2 have gather, but no scatter), and an extension all the way to 32 XMM / YMM / ZMM registers
True, using ZMM registers causes severe downclocking issues. But XMM (128-bit) and YMM (256-bit) registers are still sufficient for CPU-based tasks.
-----------------
Practical, pragmatic use of XMM registers include:
* Cuckoo Hashing (See: https://github.com/stanford-futuredata/index-baselines/blob/...) -- Have 8-bins per hash value for a total of 16-bins. Use SIMD to perform 8x comparisons at once.
* Huge number of database applications: http://www.cs.columbia.edu/~orestis/sigmod15.pdf
Bloom Filters are the most obvious "SIMD" data-structure, with applications to databases and many other tasks. Sorting networks are best implemented in SIMD.
* See this discussion: https://news.ycombinator.com/item?id=16171806
In effect, you have Base64 encoding / decoding that is faster than a PCIe x16 slot. You can encode / decode Base64 using AVX registers faster than you can even pipe data to the GPU.
---------
True, the GPU is the ultimate SIMD processor. But in many cases, the SIMD task is too fast to go to the GPU. In particular, the Base64 encoder was encoding 20GB/s, while PCIe can only transport 15.6GB/s!! The CPU is done before the data even gets to the GPU.
Ditto with latency-specific code, like Cuckoo Hashing. SIMD speeds up the overall hash-table, but you're only parallel by 8x. There's no point to actually offload a Cuckoo Hash to the GPU.
In effect, you look for "small" parallelism of size 8 to 16 or so, and that's where AVX / AVX2 / AVX512 really shines. Its too expensive to move to the GPU, but you still get a HUGE speedup when you process it on the CPU.
That seems like a good case for using integrated graphics that has access to the CPU's memory controller and so can access data directly without copying it over PCIe.
Though if you're running that kind of volume you may have PCIe as the bottleneck in any case to access the data on whatever storage/network device.
I do like AVX512 in principle, but in practice the runtime cost of using it is so high that I've offloaded anything highly parallel to other hardware (i.e. GPGPU) -- there's always work enough to keep the CPU busy anyway.
Have you found AVX512 great in practice? I'd love to hear about it!