Still, and despite writing this post which will make a lot of people express something similar to what you wrote, I consider myself an AVX-512 fan, not the other way around. It's the most important ISA extension since, well, I'm not sure: a long time (probably AVX and AVX/2 combined would have a similar impact).
It introduces a whole ton of stuff that is very powerful: full-width shuffles down to byte granularity with awesome performance, masking of every operation, often free, compress and expand operations, and a longer list at [1]. That's only from an integer angle too (what I care about).
Yeah, it's taken AVX-512 a while to get traction (the fact that generation after generation of new chips have just been Skylake client derivatives with no AVX-512 hasn't helped), but I hope we are reaching a turning point.
These transitions are something you have to deal with if you want max performance, and I think we'll come up with better models for how to make the "global" decision of whether you should be using AVX-512.
---
[1] https://branchfree.org/2019/05/29/why-ice-lake-is-important-...
The instructions are sufficiently different from AVX2 that any appropriate use is not as simple as sticking it behind a gate and using a smaller block size, it basically requires a completely separate (re)write to properly take advantage of.
I'd say yeah, you often need a rewrite of the core loop to take full advantage, but you can still more or less write AVX-style code in AVX-512 if you want, and take advantage of the width increase.
The main difference I think for most code is the way the comparison operators compare into a mask register. It would have been nice if they had just extended the existing compare into SIMD reg (0/-1 result) instructions too, to ease porting.
Why? At a higher level of abstraction, you can dispatch simd instructions at the max width available. At least, that's how I work with vectorized code. Still see gains on avx512.
IMHO the most important ISA extension since AMD64 was AES-NI, which moved a major consumer of CPU time into the also-rans.
In practice this means you could pick a few entire subsets: Legacy+SSE2 is always there for 64 bit, maybe test up to SSE4.2 for another subset. Maybe switch everything from REX to VEX if it has AVX and AVX2. That's effectively three back ends, which is manageable. With AVX-512, everything beyond AVX-512F is a la carte, and that adds unwanted complexity for instruction selection in a compiler.
Just look at all the separate AVX feature flags from CPUID:
https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_F...
Between the performance problems and complexity, I think it'll be a while until AVX-512 is attractive.
If you ignore the EOL Xeon Phi stuff (with different and incompatible ISAs), it was proceeding in a superset approach, but cascade lake and cooper lake AI extensions kind of messed that up.
Good way to visualize it:
https://github.com/InstLatx64/InstLatx64/raw/master/VennDiag...
Basically you have the SKX subset and the ICL as the big important ones in the near future, unless you care about AI, in which case Cascade Lake is like SKX + VNNI and Cooper Lake is additionally + BF16.
So in practice you'll target one of those subsets, nothing more fine-grained that that. Yes you should still test for all the required extensions, but that part is easy.
In some ways the explanation of how to simplify your view of it just makes it sound even worse.
A big question is if/when AMD starts adopting AVX-512, will they choose the same subsets that Intel did, or introduce new ones?
Basically, on the ground, it is about as complicated as say the few 128 and 256 but extensions: there are only a few sets of functionality you have to care about (2 if you don't care about AI).
It's just that within those groups Intel decided to be very fine grained about the functionality, dividing the instructions among many flags (still, in a logical way).
So instead of the new generation just supporting SSE2, say, it supports 6 new flavors of AVX-512. My claim then is that this doesn't matter much: you can just think of all of those 6 as a unit, AVX-512-ICELAKE or whatever, because there are no CPUs that support a proper subset and there probably never will be (if there is, that's fine - you'll evaluate then if it makes sense for a new codepath).
Maybe I'm not making a good case that this is same :).
I wouldn't start with the CPUID testing though.
It's more like "Why do you care about ISA features"? Usually because you are trying to choose how many code paths to support for runtime ISA-based dispatching, or how many binaries to build when you build multiple versions of a binary (which may include compile-time dispatching).
So for that planning process, you only care about a few clumps. Then your CPUID testing strategy should still test all the required extensions, for completeness, and fall back as usual. Or something like that.
Intel has made the work on OpenJDK for taking advantage of AVX when present.
[0] https://software.intel.com/en-us/cpp-compiler-developer-guid...
The trick though is _describing_ the scalar operations in the language and getting the compiler to understand how to efficiently vectorize them. I couldn't get GCC to do it at the time (GCC-5 if I recall, though we deployed with GCC-6); maybe it was just inexperience on my part. But I ended up writing the intrinsics by hand. To be quite honest it was my first dive into SIMD and I thought it was rather fun to do.
You can say -march=native -mtune=sandybridge, but there would be no point.
You can say -march=sandybridge -mtune=native, usefully. It might go slower on a real sandybridge than if tuned for it, but would still work, and would go as fast as the smaller instruction mix allows on your build machine.
Even the minimal AVX-512 ISA on any mainstream CPU (SKX) is pretty much a strict superset of AVX2.
Business side was considering whether to buy Skylake or Broadwell.
It's coming from Intel Knights Landing. Previous massively multicore Intel offerings used a Pentium core and had a different 512 bit SIMD instruction set, KL used a Silvermont Atom core and introduced AVX-512 and parts of it were implemented in the Skylake Purley platform in a desperate attempt to give KL more software. With little to no actual adaption and the desperate situation of being stuck on 14nm and overselling those capabilities because they expected most CPUs to move over to 10nm it was no surprise the Knights... chips got the axe. But, the insanity of AVX-512 having an entire menu of possible instruction subsets stayed.
I think it's also safe to assume that, given the lead time to design an ISA and integrate it into an architecture (many years), the merging of AVX-512 into Skylake wasn't done "in a desperate attempt to give KL more software".
These slides from Tom Forsyth provide lots of interesting background on the evolution of the instructions: http://tomforsyth1000.github.io/papers/LRBNI%20origins%20v4%...
AVX512-VL gives the programmer AVX512 functionality at 128/256-bit widths, if it is believed to be more beneficial than a frequency hit.