"I suspect we're about to see [Apple Silicon] move to small performance increases due to diminishing returns"

Is there any discussion of this online, I'm curious to read more.

This has already happened.

A14 is 3.1GHz while A13 is 2.66GHz. A14 is around 20% faster than A13 overall, but it is also clocked 14% higher which gives only around 6% more IPC. Power consumption all but guarantees that they won't be doing that too often.

Where does the A14/M1 suffer due to the lack of SVE? The performance of M1 is well known to be terrific, so it’s hard to characterize that as dropping the ball in my book. More like they prioritized other features instead, and they ended up creating a great processor.

I suppose they "dropped the ball" in the sense that those instructions cannot be assumed to be available, thus will not be encoded by the compiler by default. Any future processors which include the instructions may not benefit until developers recompile for the new instructions and go through the extra work required to conditionally execute when available.

That said, to get the best performance on vector math it has long been recommended to use Apple's own Accelerate.framework, which has the benefit of enabling use of their proprietary matrix math coprocessor. One can expect the framework to always take maximum advantage of the hardware wherever it runs with no extra development effort required.

I would think that Apple would make a new fat slice for ARMv9 where SVE was the baseline.

Let's say M2 has SVE. Do you:

* only use NEON to save developer time and lose performance and forward compatibility

* only use SVE to save developer time and lose backward compatibility

* Pay to support both and deal with the cost/headaches

Experience shows that AVX took years to adopt (and still isn't used for everything) because SSE2 was "good enough" and the extra costs and loss of backward compatibility weren't worth it.

If SVE were supported out of the gate, then the problem would simply never exist. Don't forget (as stated before) that there's been a huge wave of M1 buyers. People upgraded early either to get the nice features or not be left behind as Apple drops support.

Let's say you have 100M Mac users and an average of 20M are buying new machines any given year (a new machine every 5 years). The 1.5-2-year M1 wave gets 60-70M upgraders in the surge. Now sales are going to decline for a while as the remaining people stick to their update schedule (or hold on to their x86 machines until they die). Now the M2 with SVE only gets 5-10M upgraders. Does it make sense to target such a small group or wait a few years? I suspect there will be a lot of waiting.

It's even harder than that.

SVE vs NEON performance will also hugely depend on the vector length the given algorithm requires, and the stress that the instructions put on the memory subsystem.

Memory hierarchy varies by product and will likely continue to do so regardless of what M2 does.

In the end, I echo eyesee's comment, for best performance one should really use Accelerate.framework on Apple's hardware.

My point was that it probably doesn’t matter. M1 is already very fast, even without SVE. At some point you just have to decide to ship a product, even if it doesn’t have every possible feature.

Like other posters mentioned, for vector operations like this, you could be dynamically linking to a library that handles this for you in the best way for the hardware. Then when new instructions become available, you don't have to change any of your code to take advantage.

I don't think any code sensitive enough to performance to warrant SVE instructions will want to tolerate a jump in a very tight loop.

The libraries are either header-only to permit inlining or written to operate over batches of data.