The are ripple effects to consider as well. The large L1 caches of M1 (320 KB total) put capacity pressure on L2, towards larger sizes and/or away from inclusive policy. See the 12MB shared L2. Meanwhile, the narrower decode of Zen4 puts pressure on things like branch prediction accuracy & mispredict correction latency - if you predicted the wrong codepath, you can't catch up as quickly. See the large branch predictors on Zen4.
There's another factor here: to have a low latency, the L1 cache has to be indexed by the bits which don't change when translating from virtual addresses to physical addresses. That makes it harder to have a larger low-latency L1 cache when the native page size is 4KiB (AMD/Intel) instead of 16KiB (Apple M1/M2).
That is, most of the "cleverness" allowing for a larger L1 instruction cache is simply a larger page size.
(also what do you believe happens with 4k pages on M1, since it does support those, or why did the A7 have larger caches than Zen4 a decade ago, which was well before 16k pages)
Also, x86 code is compact, but not as compact as in era of 8080 [2] - here addition and multiplication require 3 bytes each, 6 bytes total. To my surprise, ARM has an add-multiply instruction and it uses just 4 bytes (instead of 8) [3].
And RISC-V uses 6 bytes because of shortened instruction for addition [4]
Of course, this simple function cannot be a replacement for proper analysis, but it seems that x86 code is not significantly denser.
Also to my great disappoitment none of those CPUs has checked overflow for arithmetic operation.
[1] https://godbolt.org/z/jsoccE5jv
[2] https://godbolt.org/z/jTMs1MEzh
Aarch64 is roughly on par for encoding density.
What? I just wrote a simple main and a couple of small helper functions. How can we be past 20 kB of .text section?
Look at the demoscene for an example of dense x86 code, especially in the sub-1K categories. They routinely achieve code densities for x86 that no compilers I know of can get close to, and AFAIK I have not seen the same happen with ARM, nor MIPS or any other well-known RISC.
3-address instructions improve code density only in situations where both source operands are needed later.
Having had to patch binaries on multiple occasions by inserting instructions, it is definitely not hard to do so for x86 as one can easily find "slack" that the compiler left behind[1], but I once had to do it for a MIPS binary, and it was definitely not easy to squeeze in the few extra instructions I needed inline; I ended up having to detour to another area with jumps instead.
Here's an old paper where the authors tried to optimise for code density manually, and you can consistently see x86 beating ARM and MIPS:
https://web.eece.maine.edu/~vweaver/papers/iccd09/iccd09_den...
[1] See https://news.ycombinator.com/item?id=15720923 for an example.
Pyrit, a ray tracing demo in 256 bytes, does all of that: https://www.pouet.net/prod.php?which=78045
You probably wouldn't want your general purpose compiler doing this sort of thing! The resulting code would be suboptimal and fragile.
Yes - we're stuck with the historically convenient collection of operations and types. Our CPUs are amazingly fast but very little smarter than decades ago.
Backwards-compatibility pretty much locks us into the same logical space. At least when people were designing new and amazing computers using actual wire (!) they felt free to implement what they thought would be useful, not just copy the existing model.
While MIPS can trap on overflow, most architectures can at least set flags for conditional branch. PowerPC OR's the overflow flag into another flag, so you can choose to check that flag for overflow first after a block of calculations.
I really want Mill to become successful. It has "NaR" (Not A Result) - like NaN but for integers. For every arithmetic instruction, there is has a variant that can set NaR. Like a NaN it is carried through the data-flow, and there's a trap first when it reaches a store or conditional instruction.
Fused multiply-add is (https://en.m.wikipedia.org/wiki/Multiply%E2%80%93accumulate_...) is extremely useful and part of IEEE754-2008.
Because of that, it would surprise me if a modern CPU with floating point wouldn’t have it.
> And RISC-V uses 6 bytes because of shortened instruction for addition
That Wikipedia page claims RISC-V has fused multiply-add, so that may be compiler inefficiency. An answer to https://stackoverflow.com/questions/57248403/why-fused-multi... says gcc can generate it at -O3)
In fact, 64-bit ARM does not have a regular MUL instruction. "MUL" is an alias to a MADD with the zero register as addend.
Flipping one parameter bit in the instruction word changes a MADD into a MSUB. That instruction is used to compute the remainder after a division. Bigger ARM cores (such as Apple M1) perform macro-op fusion of DIV and MSUB to deliver both quotient and remainder in one operation.
Anyway. I thought that was pretty clever by ARM's designers.
It's unclear to me if they will pull ahead on the perf/area game with the era of 8-wide x86 decoders coming.
The problem is that x86 has a lot of legacy that wastes the potential gain of variable length encoding. If I remember correctly, the code density of 64-bit x86 isn't really significantly better than 64-bit ARM, despite ARM dropping support for Thumb.
RISC-V is going to be interesting in that regard because with the latest revisions of the compressed instruction format, it can potentially beat both ARM and x86 on real-world code density, while still being easier to decode.
This is true in theory but I question how much it applies to current CPUs. A problem I see is that the x86 instruction set was designed in the 1970s with corresponding assumptions about instruction frequency, speed of memory, number of CPU cycles, pipelining (or lack thereof), addressing modes, etc. Even if the variable-length encoding was close to optimal back then, there's no way it's remotely optimal given all the extensions to the instruction set, the change from hand-coded assembly to compiler-generated code, and many more seismic shifts in the ISA. So I think x86's variable-length instructions now have all the downsides of complicated decoding as well as the downsides of suboptimal code density. But you also can't reshuffle all the instructions and addressing modes without breaking backward compatibility.
Zen 5 is widening the front end. My guess is with scaling coming to an end, one nice tweak with Zen 6 should be darn near the end of the performance road for a bit. Not saying the actual end, but it should be one of those sweet spots where you build a PC and it's really good for years to come.
I'm still running Raven Ridge and have no need to upgrade, but I will when I can get double the cores or more at double the IPC or more, and maybe at lower power ;-)
this article states otherwise, so how is it?
(also I'm the author of the article)
Wide x86 decode definitely has a cost, but I don't think it's the primary limiter as you make it out to be: for several x86 generations the narrowest bottleneck has been the renamer which has crept up from 3 to 6 in a glacial manner. Admittedly rename is also complicated on x86 due to memory source/destinations, 3 input instructions, 2 output instructions, etc.
IIRC Jim Keller said in some interview that modern x86 uses prediction and speculation (similar to branch prediction), and it works surprisingly well.
But fundamentally, a given chip, dedicating a given area to the task, can only begin to decode at so many positions per cycle. And the more intelligent it tries to be about where to start decoding, the longer into that cycle it needs to wait.
And one nastiness about x86 is that you have to decode pretty far into an instruction to even determine its likely length. You can’t do something like looking up the likely length in a table indexed by the first byte of an instruction.
I wonder whether modern chips have pipelined decoders.
No one would design an ISA like x86 these days. It definitely does use more power and more die area than strictly needed. It definitely does reduce performance in some applications. It definitely did take heroic engineering efforts to make x86 work well. But, all told, it just doesn't matter very much.
SMT is simultaneous multithreading, which may be more familiar under Intel's name of hyper-threading.
https://en.wikipedia.org/wiki/Simultaneous_multithreading
PPA is not the Professional Photographers of America, nor the Professional Pickleball Association, nor the Philadelphia Parking Authority, all of which came up at the top of my naïve search.
It's Power-Performance-Area.
Actually, what akvadrako described is just called multiprocessing - not SMP.
SMP refers specifically to systems with uniform latency to shared memory. So any distributed memory topology like ccNUMA w/ interconnect would not fit that definition.
Definitions are more grey, muddy and dependent on context; in that line of thought, I feel the parent you're responding to is accurate enough to not warrant criticism or even downvotes.
Like you are totally welcome to turn off SMT on most AMD and Intel motherboards (including some workstations I've seen etc - they even have "one core only" modes if you wish!). But it's a performance benefit in most situations, and it's good performance benefit relative to the cost/etc (compared to twice as many cores/cache/etc). It's just higher PPA than a non-HT design, in compute-limited scenarios.
I actually would be curious to see it on the Apple silicon too! They seem to have a very wide frontend/backend and maybe it could do a bulldozer-style switching the decoder between threads.
But in general there is indeed the sort of latency/QOS vs throughput tradeoff there. Server processing architectures go even higher on resource sharing - Sun Niagara series went to 8-wide CMT times 8 modules (8C64 threads). That's totally fine for some database workloads and they've specifically optimized their software to run well on it and utilize all the threads. And it's cheaper in your licensing too, wow how unpredicted!
POWER9 went to SMT8 on the performance cores as well. If you want to build a big fat core, it's hard to keep it utilized, and the inherent tradeoff is... just more threads.
Xeon Phi (Knight's Landing) is an interesting precedent in this thesis too! It kind of takes the bulldozer idea even further and does SMT4 and also AVX-512, but you get 54 of these P6Pro-tier SMT4 cores with AVX-512 bolted on. I know people view it as a descendant of Larrabee but it's interesting as a wide-SMT parallel processor as well, it's broadly comparable to something like Niagara in some respects.
A better analogy would be to say "Pentium is an example of an x86 CPU."
Which one? I know BOOM can technically go eight wide in so far as it's parametrizable but I suspect any BOOM backend which could support that much throughout would be a nightmare to instantiate on nearly any FPGA.
So you could argue a real mix of instructions is not going to be all 16 bit but some 16 and some 32, so the 8 is rarely achieved in practice, and also the block diagram only shows 4 decode blocks. But it can in fact peak at 8 instructions decoded per clock, so I'll call that 8 wide decode.
(You could even argue it's especially impressive, since RISC-V technically qualifies as variable-length encoding like x86, it's just that only the 16/32 instructions encoding are really in use at the moment)
A VERY general rule of thumb is that every 5th instruction is a branch, decoding more than 8 may be pointless
However VROOM! can replay 8 instructions per clock out of the trace cache .... (they're already decoded and get pushed into the pipe after the decoders)
Is VROOM primarily targeting ASICs of FPGAs? I know BOOM is primarily meant for ASICs but I feel like there is still a lot of room to build a good OoO FPGA core, particularly since us mere mortals can't get a spot on a wafer big enough to support even the smallest configs of BOOM (or presumably VROOM) since the efabless Sky130 shuttles, while "affordable", are hardly big enough to most OoO cores. I've been thinking about taking the ideas from this paper [1] and adding it to BOOM to try and make it more FPGA friendly since that seems like the most realistic way to get a very high performance custom core at this point.
I've been assuming that anyone building one would be building hand built data paths/register files/caches/TLBs/etc largely because I've come from that world
- mov (an instruction to load, store data) - Basic arithmetic instructions (add, sub, div, mul, etc.) - Bitwise operators (or, and, xor, shift, etc.) - Compare (cmp) - Branch (jz, jnz, j, ret, etc.) - There was one weird instruction I would occasionally see. It was used to zero memory (I forget its name). My guess is the compiler used it because it was faster than running creating a loop which zeroed memory.
The point is, people seem to assume ALL instructions have to be fast. The only instructions which have to be fast are the ones the compilers are using. The rest just have to work.
Making the rarely used instructions work probably takes up a negligible amount of space on a typical core. Moving these instructions to a special core would not get you much but would make old software run slower (becuase of the context switch to the other core and/or a limited number of cores it could run on).
Note that I know AMD and Intel both have a lot of guidance for compiler writers.
(1) I debugged disabled code because I was debugging optimized code and it was the only way to reliable determine why a function crashed or malfunctioned.
That’s a self-fulfilling prophecy. Compiler writers won’t use the instructions that are (relatively) slow. If, tomorrow, Intel makes ‘add’ slow, compilers will eventually start compiling addition as negation + subtraction (and AMD will become more popular)
That’s what makes hardware designers unhappy sometimes. If they ship a shiny faster CPU that requires a large overhaul of compilers, it can take time for compiler writers to catch up.
Hardware designers shouldn’t (only) aim for getting closer to the local optimum of where compiler writers are, but also for the higher peaks of where they could be.
This means to exploit your processor's full decode performance you have to reorder your instructions so the complex ones are the first in each 16-byte group, or something like that.
From the multiple "Bottlenecks in AMD Zen" sections, a common point is
>The limiting fetch rate of up to 16 bytes per clock is a very likely bottleneck for CPU- intensive code with large loops
Although admittedly, if your small hot loops fit in the uop cache that does largely mitigate the fetch/decode problem
e.g. we know about Ventana Veyron, TBA before end of year, and Tenstorrent's Ascalon, TBA 2024, competitive with Zen5 which is also 2024.
Ascalon is 8-wide, but has smaller siblings at lesser decoder width, to cover a range of uses.
There's also Rivos, MIPS and SiFive working on very high performance cores, but we know less about these efforts.
Mobile: ratio power consumption/performance to run as smooth as the latest samsung galaxy flag ship the CPU-wise worst android apps, off AC.
Server: erf, I guess that would be tons of cores with tons of ram and cache then back to full blown linux kernel/gcc/llvm compilation speed&compression/decompressing&etc.
We are talking about world domination of a worldwide royalty free/ultra stable in time ISA (oxymoron).
I understood fast as to do with roadmaps, and used it that way.
Before year end for Veyron, next year for Ascalon. Potentially, there'll be further announcements for performant microarchitectures next year.
As for performance, with Ascalon promising similar to Zen5 performance at significantly reduced power consumption, I don't think there's much reason for concern: Very performant RISC-V microarchitectures are coming.
With all these companies and their strong teams working on new RISC-V microarchitectures, there'll be plenty of choice, soon.
I am always cautious about this. Very. You have those horrible compilers in the way.
I still dunno on which risc-v microarchitecture I'll start coding assembly (wanted to start on the mango mq pro something for a keyboard firmware 100% 64bits risc-v). Finally starting to get rid of those pesky compilers and system language syntax which are unable to be stable on the long run. At best, we'll get very high level language interpreter directly written in risc-v assembly. Just need to be very cautious about macro processor usage.
I started to code some core functions directly in x86_64, and I know it is only a transition phase towards risc-v 64bits ISA (porting from 1 modern ISA to another is still much, much, less work than to code from scratch). I'll enjoy the additional register space (even with a more register space consuming load/store ISA).
BTW, we still missing hardware instruction for direct memcpy/memset/memcmp in risc-v, that said I don't know if that will be pertinent in the end.
Very good to hear, especially if it is open source and performs well (low latency and low jitter).
Ofc, this firmware would be bare metal. Not jocking around. For the moment, I am a bit scared at porting to human assembly the SOC and board init code.
The only non-open source programs I know of are elf/linux video games and the steam client.
I have an AMD CPU. Zen CPUs come with a fairly wide backend. But the frontend is what it is (especially early Zen), and without SMT it's essentially impossible to keep all those execution units fed. It's not that 8 x86 decoders wouldn't be a benefit, it's just that more decoders isn't cheap in x86 cores, each extra decoder is a serious cost.
If you compare with the big ARM cores, having a wide frontend is not a complex research problem or an impractical cost. 8 wide ARM decode is completely practical. You even have open source superscalar RISC-V cores just publicly available on Github running on FPGAs with 8 wide decode. Large frontends are (relatively) cheap and easy, if you're not x86.
So when we notice that the narrower x86 CPU's decode doesn't consume that much (a "drop in the ocean"), that's because it was designed narrower to keep the PPA reasonable! The reason I can't feed my Zen backend isn't because having a wide frontend is useless and I should just enable SMT anyways, it's because x86 makes wide decodes much less practical than competing architectures.