Apart from cache, anandtech notes https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...

  What really defines Apple’s Firestorm CPU core from other designs in the industry is just the sheer width of the microarchitecture.

"Width" enables greater parallel execution of instructions.

One of the interesting comments in that article was how they pinned limited width of x86 decode implementations on variable instruction length. There are obvious code density benefits to the x86 variable length approach (especially in immediate encoding), but I guess the need for realignment creates long critical paths on the frontend. I wonder if the more constrained variable instruction length of RV64GC (32- and 16-bit instructions only) will be able to similarly scale up to 8-wide instruction decode like Apple has been able to do with AArch64.

64bit x86 squanders it's variable length advantage by being a 40 year old design that has been extended over and over again.

Much of the opcode space is wasted with old single byte instructions that are rarely used these days. There are REX prefix bytes required everywhere to access all 16 registers. Modern instructions that are used all the time are hidden away behind prefix bytes.

64bit ARM is a complete redesign of the ARM instruction encoding, and they put a lot of thought into using the instruction space optimally. 32bit ARM also wasted a lot of it's instruction encoding space, but 64bit ARM is a massive improvement. Despite requiring roughly 10% extra instructions on average, the average arm64 is around the same size as the average x86 binary.

Immediates aren't a huge problem. All 32bit immediates and many 64bit immediates can be encoded in 1-2 instructions. Anything that can't should probably just use a PC relative load.

IMO, the much simpler instruction decoding massively outweighs the need for slightly more load bandwidth.

I've heard that there's other benefits, like the fact that memory RMW instructions are essentially a fairly clean way to address physical registers without using any architectural registers.

I'd love to see what a CISC-V ISA without the 40 years of baggage looks like (like jeeze, the hlt instruction gets a single byte on x86?)

Agreed. AMD didn't throw enough away when they moved X86 to 64bits. They also didn't add enough registers. Variable length CISC is fine. They should steal the bit manipulation instructions from ARMv8 and the vector extensions from RISC-V.

They won't.

"Variable length CISC is fine." Well, actually, the Anandtech article points out:

"Other contemporary designs such as AMD’s Zen(1 through 3) and Intel’s µarch’s, x86 CPUs today still only feature a 4-wide decoder designs (Intel is 1+4) that is seemingly limited from going wider at this point in time due to the ISA’s inherent variable instruction length nature, making designing decoders that are able to deal with aspect of the architecture more difficult compared to the ARM ISA’s fixed-length instructions."

Given the complexity of x86, it's amazing that Intel+AMD have gotten 4-wide decoders. But the M1 has 8-wide and if they want to go wider, it's linear rather than quadratic in complexity.

I think they did a fine job considering all the constraints. The ISA was created in 1999, but the first widely used 64-bit OS (vista) didn't release until 7 years later. In truth, I doubt 64-bit systems became more than half of all systems until a couple years after the release of Window 7 in 2009 (a full decade later).

Every transistor used for x86_64 that couldn't also be used for x86 was a competitive liability (increasing size, power, R&D, etc without any real payoff). I think their decisions make a lot of sense given all the constraints.

I agree that they did a fine job given the constraints at the time. But by getting rid of x87, ... they would have used less size, power, R&D. SSE was circa 1999 and they could have just supported that.

They being AMD. The counterargument being that if AMD had been too aggressive, Intel could have done something more conservative. However, then there's the cross licensing agreement ... Arggh.

They made SSE and SSE2 extensions part of the core instruction set in AMD64.

But they didn't remove x87. It's not really used, compilers only emit it when code asks for a long double.

Personally, I do think they should have banned x87 from 64bit code. But it wouldn't have allowed them to remove the x87 units from the chip, as every AMD64 chip to this day still supports 32bit compatibility mode, and regularly uses it.

A code density prioritizing modern ISA is something that I've thought a lot about too. Would be an interesting possibility in the embedded space. I think that there are certain tricks that can be used to speed up the realignment problem, too.

Interesting point. I guess as cache sizes go up instruction density becomes less important. Also Arm abandoned Thumb - which was relatively easy to decode - for AArch64 and I guess they must have done quite a lot of analysis before doing so.

Thank you for the link. That article says the cpu has a 630 op reorder buffer. Does that mean it has to throw out up to 630 instructions on a mispredict? That sounds huge, so I wonder if I’m misunderstanding.

Imagine a cpu that executed this sequence of instructions:

    r1 = r2 * r3
    r2 = r3 * r4
    r3 = r4 * r5
    r4 = r5 * r6

On a first glance it might seem that the execution order matters because if you reorder lines in this source coxe the meaning indeed changes (e.g. the first instructions depends on an input in r2 but r2 gets a new value in the second instruction).

But if you rewrite the names of the registers (variables) and use a larger set of registers (variables) then you can express the same semantics without overwriting any registers. If you do so you can run the operations in parallel on different execution units within the same core.

The ability to run multiple functions at the same time is called "multiple (or wide) issue".

There are different kinds of functional units inside a cpu. For example, some can do basic arithmetic but not divide. At any given time, some functional units will be busy or free. The amount of effective parallelism present in the input stream you can effectively use is bound on whether you can successfully map your instructions into functional units that are free at that time.

Often, you while cannot run a given instruction because for example the multiplier unit is busy, the next instruction could be run because it depends on the divider functional unit, which at that time is free.

The ability to execute instructions out of order is called, well, "out of order execution". It depends on the ability to rename registers and keep track of actual data dependencies. The cpu needs some memory to keep track of that data.

It's unrelated pipelining (which I could give a try explaining too if anybody cares to hear a similar explanation)

Nice explanation. Wouldn’t mind reading more.

I'll try. On pipelining:

Most operation above a certain level of complexity can be broken down in individual smaller steps that have to happen in order and each step takes some time. Once one such step is done, the next step is performed etc.

If the logic for each of those steps is implemented in hardware, once a step is done executing, the logic just sits there idle, waiting for the next operation (composed of smaller steps) to be executed.

Pipelining is a technique that allows to make use of that otherwise idle logic to start performing the first step of the next operation.

For example, let's consider this small program:

    r3 = r1 + r2
    r2 = r3 + r1
    r1 = r3 + r4

A hypothetical CPU would execute this instruction in 12 clock cycles:

    load instruction 1
    decode instruction 1
    perform r1 + r2
    write result in r3

    load instruction 1
    decode instruction 2
    perform r3 + r1
    write result in r2

    load instruction 1
    decode instruction 3
    perform r3 + r4
    write result in r1

A pipelined execution looks like this:

    load instruction 1 
    decode instruction 1 | load instruction 1  
    perform r1 + r2      | decode instruction 2 | load instruction 3
    write result in r3   | perform r3 + r1      | decode instruction 3
                           write result in r2   | perform r3 + r4
                                                  write result in r1

and completes in 6 cycles. Once the pipeline is running at full capacity, it produces 1 result per clock cycle, achieving a 4x speedup over the naive approach above.

This example architecture has a pipeline depth of 4.

The execution is still strictly in order. The idea is that you can start doing some work for the next instruction before the previous one is fully completed.

For example, you can decode the next instruction after you decoded the current one. Also, you can perform a computation on the next instruction as soon as you have computed the results of the previous one, even before you actually have written the results in the actual destination register, provided there is additional logic that ships the result of the current operation as an operand of the arithmetic unit for the next cycle.

The maximum duration of each step is bound by the clock period. The faster the clock, the less time you have for a single step. The deeper the pipeline, the faster the clock can tick and still produce one operation per clock tick. We'll see next some of the many things that prevents us from just riding this idea to the extreme and deepening the pipeline to ludicrous amounts and harness ludicrous clock speeds.

You may have noticed that this dummy architecture here has been carefully designed so that the next instruction can consume the result of the previous instruction if the instruction stream so requires.

Imagine a different architecture that is divided in 6 steps, where the + operation itself is pipelined in two steps.

    load instruction 1
    decode instruction 1
    access r1, r2
    start +
    continue +
    write result in r3

    load instruction 2
    decode instruction 2
    access r3, r1
    start +
    continue +
    write result in r2

    load instruction 3
    decode instruction 3
    access r3, r4
    start +
    continue +
    write result in r1


    load instruction 1
    decode instruction 1 | load instruction 1 
    access r1, r2        | decode instruction 2 | load instruction 3
    start +              | access r3, r1        | decode instruction 3
    continue +           | start +  (HAZARD!)   | access r3, r4
    write result in r3   | continue +           | start +
                           write result in r2   | continue +
                                                  write result in r1

Now, here the second instructions depends on r3 which is has not yet finished computing! This is known as a data hazard. A common way out is to introduce a pipeline "stall", i.e. a no-operation step is injected in the pipeline to resolve the hazard.

    decode instruction 1
    access r1, r2        | decode instruction 2
    start +              | access r3, r1        | decode instruction 3
    continue +           | nop                  | nop
    write result in r3   | start +              | access r3, r4
                           continue +           | start +
                           write result in r2   | continue +
                                                  write result in r1

This short stall is not a full pipeline flush. Ok, so we saw a data hazard, but so far all this has been pretty straightforward.

So far we were looking at a linear instruction stream. Let's see what happens when you have a branch:

    r3 = r1 + r2
    call func1
    r1 = r3 + r4

where func1 is:

    r2 = r3 + r1
    return

Let's see what our dummy 4-stage pipeline machine would do:

    load instruction 1                   
    decode instruction 1 | load instruction 2  
    perform r1 + r2      | decode instruction 2    | load instruction 3
    write result in r3   | perform pc + 1          | decode instruction 3 (HAZARD!!)
                         | write pc=func1, ra=pc+1 | perform r3 + r4      (^^^^^^^^)
                                                   | write result in r1   (^^^^^^^^)

calling a function basically means setting the program counter (aka instruction pointer) register to point to a different location instead of the next instruction. In order to return from the function you also need to save the return address somewhere (some CPUs use a "link" register, some use the stack, here I used a return address "ra" register); the return address is the address of the next instruction in the instruction stream before jumping to the function body.

As you can see, once we set the program counter and tell the CPU that the program flow continues from another place, the work the CPU has started doing in the pipeline becomes invalid.

    load instruction 1
    decode instruction 1 | load instruction 2
    perform r1 + r2      | decode instruction 2    | load instruction 3
    write result in r3   | perform pc+1, &func1    | nop
                         | write pc=func1, ra=pc+1 | load ins func1.1
                                                   | decode ins func1.1 | load ins func1.2
                                                   | perform r3 + r1    | decode ins func1.2
                                                   | write result in r2 | read ra
                                                                        | write pc=ra
                                                  

Jumping to a different address thus incurs in additional delay, because we don't know yet where the jump is taking us until we have decoded the instruction and performed the necessary control flow adjustments.

In our dummy example this adds 2 cycle latency. Not all is lost since we can still do a little bit of pipelining, but for all intents and purposes this is a pipeline flush.

Some architectures (especially the older RISCs but it's still quite common in DSPs) work around this problem by introducing one or more "branch delay slots"; these are instructions that appear physically after a call/jump/branch instruction but get executed before the branch is actually taken.

A equivalent program for such an architecture would look like:

    call func1
    r3 = r1 + r2
    r1 = r3 + r4

where func1 is:

    return
    r2 = r3 + r1

And be executed as

    load instruction 1
    decode instruction 1    | load instruction 2 
    perform pc+1, &func     | decode instruction 2 | load ins func1.1   |
    write pc=func1, ra=pc+1 | perform r1 + r2      | decode ins func1.1 | load ins func1.2
                              write result in r3   | read ra            | decode ins func1.2
                                                   | write pc=ra        | perform r3 + r1
                                                                        | write r2

By avoiding the pipeline flush we can now complete the same task in 7 cycles compared to the 9 cycles shown above. Branch delay slots are no panacea though and modern CPU have largely moved away from them, favouring other techniques to predict whether and where a branch is likely to happen. I choose to mention branch delay slots in order to illustrate that pipeline stalls are not inherently necessary whenever branches are involved.

Stalls are ultimately caused by data dependencies. The program counter is just data, on which the instruction stream itself depends on.

(In the real world obviously things get much more complicated but you get the idea)

Does this also refer to physical die width? As in, it also makes the manufacturing more expensive as yields go down due to any small defect?

No, the size of the die isn't really related to the internal structure of a CPU core.

I would quote Orwell here: "Who Controls the Past Controls the Future".

It is about lack of control over legacy killing the progress. The power of Intel is "proven" legacy, but building completely new architecture is expensive and prone to adoption failure (Itanium?) when Intel has no control over OS (Windows), apps or virtualization software.

Then Apple has control over all that: they can make hardware (scaled up mobile HW), change their OS to support it (iOS bits), support past apps (Rosetta), enforce support in new apps (compiler, LLVM, certification).

Itanium was a really bad architecture for the time (and possibly still is). It was a bet that sufficiently smart compilers would be available, but they weren’t available in time. They aren’t even available today 20 years down the line in a form that would have justified Itanium if they were available then.

It wouldn’t have mattered if Intel controlled the OS or not - it just wasn’t competitive. There’s nothing you can learn from it about Apple - who delivered an ultra competitive, likely even when emulating Intel architecture in software.

I think that the ARMv7 -> ARMv8/Aarch64 move is interesting in this context.

Arm has swapped a large part of its application user base from 32 bit to a completely incompatible but more modern 64 bit ISA and Apple has cast off the 32bit legacy completely due to its ability to control the whole stack.

Apple is now Full-Stack. It's turtles all the way down to the CPU!

This is not really the case. Most Intel servers run linux, Intel has nearly complete control over linux in a practical sense. They can put developers in it and update the OS and unless its a very user hostile feature it will go into the mainline kernel and be everywhere. The same goes for compiler and so on.

Intel could easily open up their boot and make the store there much better.

The don't quite have the vertical control of Apple, but to say they are fundamentally hindered by their legacy is quite questionable.

The fucked up improving their manufacturing.

Unlikely. There is more Linux devices running as Android on ARM than any other processors. You I can run Termux on ARM Android with complete set of of GNU tools. Also ARM is supported natively by Debian and Ubuntu [1].

Once devs will sort out their new shiny ARM Macs whoes (Docker, other tooling) I smell there may be ARM server market rising soon.

[1] https://help.ubuntu.com/lts/installation-guide/armhf/ch02s01...

Amazon already offers EC2 A1 (ARM64) instances available, and I'm sure Nvidia's purchase of ARM is going to bear fruit in the data center.

AMD's Epyc chips are pretty awesome though too so I don't think x64 is on it's death bed.. but it should be an interesting couple of years watching this space heat up.

Intel's current design are 1-3 years late depending on which roadmap and sector you are counting. Caused by their 10nm delay.

Intel had Tigerlake / WillowCove Design but they cant ship it. They also have Golden Cove. If you have put the design they were suppose to be using now, on a IPC basis Intel might still be in the lead.

It has really been cascading process delays dating back to 2014 when 14 nm got pushed back when Broadwell was supposed to launch, too.

What takes time to bring a chip to market? As in the question, if you know exactly what needed to be designed (in concept), how long would it take you to have a fab turning out units?

Is it the work to figure out how to lay down the component blocks on the die and all the circuit routing? Is it figuring out how to assemble the power-efficient components from the Arm library? Does that take years? Is it the instruction set? Is it the planning for manufacturing capacity?

I'm just curious to know what the actual work and division of labor / time is to "make a chip".

I would love to know for example:

-- Initial design, 10 people, 1 year

-- Prototype runs, 5 people, 6 months

-- Final design, 5 people, 1 year

-- etc etc.

-- Acquiring fab capacity / manufacturing hardware, 10 people, 1 year

-- etc etc.

But sounds like there's no way Intel or others can just "leapfrog" even if they knew what needed to be done and could abandon their prior choices.

>But sounds like there's no way Intel or others can just "leapfrog" even if they knew what needed to be done and could abandon their prior choices.

At best It take one year to build a Fab. It takes another one just to plan to built it. That is assuming you get the regulation approval with zero delay.

Building a Fab means absolutely nothing if you cant get the process to yield. Look at Samsung Foundry in the past 3- 5 years.

Intel's Leapfrog chances was their Intel Custom Foundry. Had they not mess that one up they would have a Fighting chance. Even if they had a Design that Leapfrog Apple's M1 today, it would still be let down by their 10nm SuperFin Process.

What about other ARM chips?

Other ARM chips are aiming for smartphone/tablet and are more than enough for this. No one using samsung latest top phone is thinking "if only that phone was 15% faster ! this is a major issue".

No one makes ARM chips for laptop, because of the chicken and egg issue. Windows on ARM going mainline could change it.

Amazon’s Graviton2 (originally from Annapurna Labs) is apparently delivering significant efficiency on the server side. Laptops will be there soon enough even without windows.

Hopefully without windows. It would be nice if we could get our own little *nix corner.

We've seen some of this already, for example with Microsoft's SQ1/SQ2 SoC manufactured with Qualcomm for the Surface Pro X series of laptops. They already emulate x86 well enough to run apps compiled for 32bit x86, and they should be coming out with 64bit support soon.

Once Microsoft is shipping these layers within Windows, it's only a matter of time before the use of ARM CPUs in more laptops takes off.

Does windows on ARM support emulation of x86 apps? Probably the biggest advantage for windows right now is tat you can run apps from 20 years ago perfectly fine, and of course the large library of playable games built over the years.

So basically given the same process, all companies have almost a similar offering?

Disclaimer: I’m a chip design enthusiast and not a professional, but this is how I understand things right now.

If you read the anandtech article describing the A14/M1 chips, it seems that the ARM ISA lets Apple do some things that are either exceptionally hard or impossible on x86, like making individual cores much “wider” in terms of instruction decode, allowing the A14/M1 cores to do significant parallel execution. This allows their cores to be significantly more efficient in terms of instructions per clock cycle. To be clear this isn’t just the benefits of ARM - Apple is far ahead of other ARM designers as well. On the other hand, the wider the execution pipeline, generally the slower the cores have to be clocked (Im actually not sure why this is, to be honest). This is part of how Intel is able to produce chips in the 5.0+ GHz range.

So it’s not that every designer converges to the same design (internally, they’re quite different). Instead, what’s happening is that all the relevant trade offs (pipeline width and depth, ISA, fab tech, power targets, etc) are leaving competitors within an order of magnitude in performance. And the question of who’s going to take the lead is a question of who’s executing best on all these fronts, and who’s currently got a lot of talent working on areas that will see continued improvement.

But whatever happens, I wouldn’t bet against Apple. Their team has been delivering improvements like clockwork, year over year for quite a long time now.