1. https://www.mono-project.com/news/2016/09/12/arm64-icache/
> The OS can install an illegal-opcode exception handler. When a process is first run on the small CPU, the unsupported opcode will raise the exception. The exception handler can simply set the processor affinity of the process to the main CPU and put it to sleep. The OS will handle it like it normally would - putting the process in the run queue of the affined processor.
Linux already tracks how long ago a task used AVX-512. I assume the same mechanism could be used to track AVX as well.
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/...
At least this only has to be done once, and frankly these AVX instructions are slow initially anyway.
FWIW I believe only some of them, and those are just some (or all?) AVX-512 instructions. I think AVX2 is implemented on the main die and doesn't have to be powered up first.
AVX (256 bit) instructions also suffer a penalty. Both the 256 and 512 bit instructions resulted in a slowdown for 9 microseconds featuring a quarter the instructions per clock, but the 512 bit instructions resulted in an additional penalty of 11 microseconds without executing instructions.
The first penalty was associated with voltage, and the second with frequency. Heavier 256 bit instructions would probably have resulted in the frequency transaction as well.
And generally speaking, when an application first starts issuing SIMD instructions, that's probably not the a great time to be interrupting it even if it only needs to happen once.
the issue is that once you have an active process using a feature only available on the larger cores, you can't shut off the larger cores to save power without paying a large latency to wake up that process.
Yes that's the idea.
> Which would be slow I would assume.
How expensive do you think a trap is? It takes about the order of 10 billionths of a second.
> If you never switched back, wouldn't any process using advanced vectorized instructions (like anything using a decent libc) be permanently pinned to the large core?
I think you can switch back next time you schedule.
are you sure about that? I would expect at least a couple of orders of magnitude more just for the userspace->kernel transition.
edit: for what is worth, a syscall it takes 250ns on my (admittedly vintage) machine. That's using the lowlatency sysenter path. An interrupt is probably going to cost more.
Anyway the cost of scheduling on another core is going to dwarf that.
edit2: for reference, this was a Sandy Bridge turboing at 3.5 Ghz during the test. With spectre mitigations on (which is going to be a good chunk of that overhead).
Ok, yes, then we're on the same page. I would still think that would be slow? You'd need a full transition-to-kernel and context switch before you could execute again, which AFAIK would take at least microseconds…unless you think there would be a faster path to resume execution?
No that's around 30 ns on modern hardware I believe.
[1] well, I hope the larger cores support a superset of the smaller cores, otherwise it would really be insane.
The only way to make big.LITTLE work and allow task migration between them is to have them support the same instruction set exactly. So, I suspect that that is why AVX512 was removed from the big core here, and I suspect the atom cores aren't exactly garden variety either. They probably support most other extensions of modern x86 CPUs, at a slow speed. For example, AVX2 can be microcoded by running multiple pieces of the data through the ALU, one at a time.
I don’t think users would be happy if it were a lottery whether their code runs slow, using microcode instructions, or fast, on the better CPU, especially given that, I guess, the difference in instruction set is for vector instructions (because adding those is what’s makes a CPU big nowadays)
So, performance oriented code will pin its performance-critical threads to the fast CPU. Now, the question is: what code won’t try to claim the faster CPU, given that benchmarks typically run programs while there is no contention from other programs?
If the microcoded vector ops cause your code to run slow, then you will be consuming lots of CPU time, and should be migrated to the big core anyway, right?
Do apps have CCX pinning logic for Ryzen? No? Why would they add it for an Intel processor that probably won't even be sold in volume.
[1]: `target_clones` -> https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute...
I'm not a programmer, but doesn't compiled code often offer multiple codepaths, checking flags at runtime? So if your code runs on the big core, it finds avx256 is supported, and uses the avx256 codepath. If it runs on the little core, it finds avx256 is not supported and it takes a legacy floating point codepath instead.
It depends a lot on which instructions differ and how the feature flags work, of course. They might have set it up in a way that's fine.
The only complication here would be if they have differing extensions like AVX512, but that's easily solved by the OS by just advertising the common baseline. Nothing about this looks difficult to support?
What does that mean? Advertise to whom? The process/process loader? Does it mean that I can’t compile with -mavx2 anymore? What if I do?
The extensions are the whole problem.
Runtime detection is the process querying what extensions are available, and then selectively using those. You adjust what the query returns to only return the common set.
Runtime detection has been a pretty standard thing for well over a decade now - it's how we all manage to run the same compiled binaries over the years despite variability in SSE & AVX support. You don't download different versions of Chrome/Photoshop/Gimp/Premiere/Blender/Whatever compiled for different CPU micro-architectures, do you? You might if you run Gentoo I suppose, but that'd be about it.
> Does it mean that I can’t compile with -mavx2 anymore?
You already can't if you're shipping binaries to users unless you only support Skylake & newer? There's a lot of CPUs currently in use that don't support AVX2. So... you either already have this problem and you're familiar with it, or you're not doing this and it's moot.
Except that they almost always do this runtime detection once, on startup, and then choose/thunk codepaths accordingly. If the OS just happens to start my avx2 process on a little core (and how is it going to know better?), that's going to turn off all of my optimizations, regardless of where the process subsequently gets migrated to.
You already can't if you're shipping binaries to users unless you only support Skylake & newer? There's a lot of CPUs currently in use that don't support AVX2. So... you either already have this problem and you're familiar with it, or you're not doing this and it's moot.
Except nobody in 30 years of x86 dev expects to get a different answer from CPUID during runtime.
That's partly what they did. From the article: "One thing we can confirm in advance – the Sunny Cove does not appear to be AVX-512 enabled."
Maybe they also fused off AVX & AVX2 support in the Sunny Cove core as well, we'll see.
And disabling AVX in cores that otherwise support it is already a common thing - see the Pentium & Celeron lineups that Intel currently sells. They don't have AVX/AVX2, even though the cores inside them definitely could offer it.
Edit: Also: AVX2 is a lot older than Skylake. You're probably thinking of AVX512.
But it looks like Intel is doing this anyway, as Sunny Cove in this application has had its AVX-512 removed anyway: "One thing we can confirm in advance – the Sunny Cove does not appear to be AVX-512 enabled."
But this shoots the big core in the foot. You run at the lowest common denominator and the big core doesn't have the advantage of higher clocks. And this might just lead to a lot of software that only runs on the big core.
Imagine you wanted to fly around between multiple points but if you want the option to ever switch to a bus then your plane will circle around each airport until it's as slow as the bus. Or you can opt for "plane only".
A better mix would have been simply using lower clocked, lower powered cores of the same type or really close derivatives of the big core where the manufacturing process and clocks are what keep power low. Not a mix of Ice Lake and Atom. But right now Intel would throw everything at the wall to see what sticks.
And it seems like a good way for developers to make sure their software stays on the big core.
AVX2/AVX-512 is great in HPC workloads, for example. But nobody is running an HPC workload on a 7w netbook, now are they?
What is useful on Xeon and what is useful on Atom are different. This is an Atom-class SoC used in Atom-class applications, not a Xeon-class one.
> You already can't if you're shipping binaries to users unless you only support Skylake & newer?
Huh? My 6+ years old gaming PC supports AVX2, and it definitely doesn't have a "Skylake & newer" CPU!
AVX2 support started at Haswell, or Intel core 3rd generation. We're at gen 10 now.
While there certainly are still a lot of systems without AVX2 support, new games (and other performance hungry software) requiring it would not be completely unreasonable.
(I recall a recent news story about how unawareness of this among people writing or documenting compilers has started to cause problems.)
> (I recall a recent news story about how unawareness of this among people writing or documenting compilers has started to cause problems.)
You can't blame them!
Which they do, Atom tops out at SSE4.2.
SSE itself tops out at 4.2, but Tremont does support the newer SHA extensions.
AVX appears to be the only missing thing, but AVX isn't even standard across Intel's other lines, either. The Pentium line doesn't support AVX either, for example, even though they are using Skylake & newer micro-architectures.
So you currently can't assume AVX support, and you still won't be able to assume AVX support. Why does this matter?
Operating systems move threads between cores. If those cores support different features, threads that are migrated to low-feature cores might experience illegal instruction traps despite correctly checking for instruction features.
For this to work, there would need to be a small set of undemanding tasks that account for a lot of machine time. Feeding video to the GPU? All sorts of GUI compositing and housekeeping? Handling network connections in the browser?
I don't think this is a good explanation, but it's fun to think about.
But what happens when the user opens up a photo viewer app and suddenly wants those photos to be synced right now?
If your code is running on a recent iPhone -- the heterogeneous-core platform I'm most familiar with -- then the answer is that the kernel will immediately detect the priority inversion when a foreground process does an IPC syscall, bump up the priority of the no-longer-background process, probably migrate it to the fastest core available, and make it run ASAP. Then, once the process no longer has foreground work to do, it can go back to more power-efficient scheduling.
This kind of pattern is super common, and it would be way more annoying and perilous to try to split tasks into always-foreground and always-background.
Even just looking at a browser, I might have half a dozen generic tab processes open, each using a small amount of CPU. But then I navigate one to a game, and I want that particular tab to get near-exclusive access to the big core while the others use only the little cores.
All that can be done with SSE and probably with x87 instructions, but, still, software will try to pick the best option.
If a binary does that, the OS can just not migrate that binary across cores.
The only mechanism I can come up with is to detect illegal instruction traps on small cores, then flag the thread as big-core-only and re-start the execution at the bad instruction. That's not ideal but maybe workable.
(If it traps on the big core, too, it's just a bad instruction and SIGILL is raised to userspace like usual.)
Are the little cores’ instructions at least a complete subset of the big cores’? Are we going to have some ridiculous situations where the little cores are completely pegged but the OS can’t migrate their processes off to the big core?
Or do kernel programmers need to start chasing Intel and trap/software implement every single future AVX8192AESNISSSE instruction Intel jams into future instruction sets to provide Xeon market differentiation?