I'm guess that your assembly code is RISC-V with the Zba extension. Is the non-Zba version worse than Arm64?

Compiling your function with Godbolt, I get:

  RISC-V (no Zba) Clang - 7 instructions - https://godbolt.org/z/7znnrzxKq
  Arm64 Clang -           7 instructions - https://godbolt.org/z/Trv8scxad

Annoyingly I can't see the code size for the Arm64 case because no output is generated if I tick the "Compile to binary" option in "Output". I have to use GCC instead:

  RISC-V (no Zba) Clang - 20 bytes - https://godbolt.org/z/eWfPaorcj
  Arm64 GCC             - 24 bytes - https://godbolt.org/z/bzsPzov5h

EDIT: Hmm, I seem to have picked a bad example. Try this one:

    int get(int *base, unsigned index) {return base[index];}

Arm64:

    update:
        ldr     w0, [x0, w1, uxtw 2]
        ret

RV64GC (vanilla):

    update:
        slli    a5,a1,32
        srli    a1,a5,30
        add     a0,a0,a1
        lw      a0,0(a0)
        ret

RV64GC+Zba:

    update:
        sh2add.uw  a0,a1,a0
        lw      a0,0(a0)
        ret

Arm64 is able to do some indexed loads in a single instruction that might take two in RISC-V w/Zba (and up to 4+ in regular RISC-V). However, calling that a win for Arm64 is not so clear as the more complicated addressing modes could become a critical timing path and/or require an extra pipeline stage. However, as a first approximation, for a superscalar dynamically scheduled implementation, fewer ops is better so I would say it's a slight win.

I don't understand the obsession with bytes. 25% fewer bytes has only very marginally impact on a high-performance implementation and the variable length encoding has some horrendous complications (which is probably why Arm64 _dropped_ variable length instructions). Including compressed instruction in the Unix profile was the biggest mistake RISC-V did and I'll die on that hill.

ADD: Don't forget that every 32-bit instruction is currently wasting the lower two bits to allow for compressed, thus any gain from compress must be offset by the 6.25% tax that is forced upon it.

Why using an unsigned? It is obvious here that RISC-V without Zba takes 4 instructions because it manages special cases related to unsigned.

If you use a simple int for index:

  slli  a1,a1,2
  add   a0,a0,a1
  lw    a0,0(a0)

And isolating this code in a small function puts constraints on register allocation, but if we remove this constraint then we can write:

  slli  a1,a1,2
  add   a1,a1,a0
  lw    a1,0(a1)

Which is very suitable for macro-op fusion and C extension

> Including compressed instruction in the Unix profile was the biggest mistake RISC-V did and I'll die on that hill.

This is so wrong. The C extension is one of the great strengths of RISC-V, it is easy to decode, very suitable for macro-op fusion, and it gives a huge boost in code density

Indeed why use unsigned? Go take a look at a lot of C code (hint, look at SPEC benchmarks). They do that. Decades of pointer == int == unsigned have let to a lot of horrific code. But we still compile it.

The sins of the original RISC-V was spending too much time looking at RV32 and not realizing how big a problem this is in practice. Zba (slipped in as it’s not really “bit manipulation”) fixes the worst of this.

ADD: The problem in this HN thread is the same reason we got compressed it in the first place. The vast majority of people aren’t doing high performance wide implementation so the true cost isn’t widely appreciated. The people holding the decision power certainly didn’t understand it. I really think you have to live it to understand it.

> Go take a look at a lot of C code (hint, look at SPEC benchmarks). They do that.

This does not really justify isolating that snippet if you admit yourself it's a bad one.

> The sins of the original RISC-V was spending too much time looking at RV32 and not realizing how big a problem this is in practice.

But indeed, my previous message shows that even without Zba the problem is erased by a good register allocation and macro-op fusion.

I think you are trying too hard to find special cases that "trick" RISC-V, you didn't even pay attention to the use of unsigned which is non-optimal (unsigned has an undesirable overflow semantic here).

Iirc compressed instructions are the thing that costs 2 bits per 32 and was criticised as overfitted to naive compiler output. Am I thinking of something else?

Yes, but RISC-V still has a lot of encoding-space free and the benefit of C extension is huge. It's a trade-off.

I don't think RISC-V is perfect or universal, but on this point they do a pretty good job compared to other ISAs

You say that the benefit is 'huge' but why should I care about code density on a modern CPU with gigabytes of memory and large caches?

From a performance perspective what is the evidence that this actually provides an advantage?

To clarify, I'm not saying that RISC-V code density of C extension is a big advantage over its competitors, but it is a huge benefit for RISC-V.

You are right, code density is perhaps not that critical today.

And it is difficult to quantify its relevance as code density is always related to other variables such as instruction expressiveness, numbers of uop emitted, etc.

But I still think that code density is important for RISC-V. Because RISC-V philosophy to reach high performance is to use very simple instructions that can be combined together and take advantage of macro-op fusion. I think RISC-V without macro-op fusion can't reach the performance of other ISA.

But RISC-V with all these simple and not very expressive instructions and without C extension has a pretty bad code density which could cost a lot because it is not at the competitors' level.

So if we think of RISC-V as a macro-op fusion oriented ISA, then the C extension becomes important to be competitive.

I don't know what is better between a "macro-fusion" oriented arch or a "complex-instruction" oriented arch, future will tell us.

Thanks for clarifying - interesting to see a different philosophy being tried.

Because the fastest cache levels are tiny, even in the largest and most advanced CPU's. There's plenty of evidence for the performance benefits of improved density and terseness in both code and data.

The M1 has a 192k instruction cache for performance cores which is not ‘tiny’.

If there is lots of evidence for the performance benefits of improved density vs the alternative of fixed instruction width in real world CPUs then I’m sure you’ll be able to cite it.

>The M1 has a 192k instruction cache for performance cores which is not ‘tiny’.

ARMv8 and ARMv9 have poor code density. These cache are large as a workaround to that.

This isn't free, as besides making the die larger (and thus lower yields), the L1's clock speed is limited due to its size.

32 bit, 32 registers, three register code. So add r0 r1 r2 spends fifteen bits on identifying which register to use then another two on the compressed ISA. That's half the encoding space gone before identifying the op. Never thought I'd want fewer registers but here we are.

If the compressed extension is great in practice it might be a win. If the early criticism of overfit to gcc -O0 proves sound and in practice compilers don't emit it then it was an expensive experiment.

The encoding space is not the number of bits used to encode an instruction, the encoding space is the ratio of values that encode a valid instructions over the total number of instructions that could be encoded.

As an example, in an ISA with 8-bits fixed instruction length and 8 registers (reg index encoded on 3 bit):

If the last 2 bits are the opcode and we define 2 instructions (eg. AND, XOR) that manipulate 2 registers (3 bits + 3 bits), then instruction word values 0b00_000_000 to 0b01_111_111 might encode these two instructions (ignore "_" they are separators).

Therefore, instruction word values from 0b10_000_000 to 0b11_111_111 remain free, which represents half of the encoding space. So half of the encoding space remains free.

This means we still have room to put new instructions.

Similarly, RISC-V valid instructions use almost all the available bits, but there is still room in the encoding space because some opcodes remain free.

> 25% fewer bytes has only very marginally impact on a high-performance implementation

Instruction cache doesn't come for free, and is usually pretty small on most shipping processors. It's not a big deal for smaller benchmarks, but in real-world programs this can become a problem.

I am obviously aware and I'm here to tell you that the overhead of variable length instructions matters more. Arm agrees. M1 has a 192 KiB I$ btw.

ADD: had RISC-V just disallowed instructions from spanning cache lines and disallowing jumping into the middle of instructions then almost all of the issues would have gone away. Sigh.

I actually had Apple's chips in mind when talking about "most shipping processors" because they have historically invested heavily in their caches and reaped benefits from it. But not all the world's an M1, and also I'll have you know that Apple themselves cares very much about their code size, even with their large caches. Don't go wasting it for no reason!

(I should also note that I am pretty on board with you with regards to variable-length instructions, this is just independent of that.)

Variable instruction sizes have a cost, but with only 2 instruction sizes like current RISC-V that cost remains very low as long as we don't have to decode a very large number of instructions each cycle, and it gives a huge code density advantage.

This biggest issue is one instruction spanning two cache lines, and even two pages. This means a bunch of tricky cases that is the source of bugs and overheads.

It also means you cannot tell instruction boundaries until you directly fetch instructions, so you cannot do any predecode in the cache that would help you figure out dependencies, branch targets, etc. These things matter when you are trying to fetch 8+ instructions per cycle.

> This biggest issue is one instruction spanning two cache lines

Even with fixed (32 bit) instruction lengths aligned on 32 bit, when we have to decode a group of 8 instructions you are facing this kind of issue.

So you either have to cut the instruction group (and thus not take full advantage of the 8 way decoder) or you have to implement a more complex prefetch with a longer pipeline. And these special cases can be handled in these pipeline stages.

> It also means you cannot tell instruction boundaries until you directly fetch instructions

I mean, AMD does that on x86, with 14 instruction lengths.

It can be done for RISC-V, it's much cheaper than x86, and it takes significantly less surface area than a bigger cache to compensate.

Given the compression stuff is an extension (and so far as I can tell the 16-bit alignment for 32-bit instructions that can result in that sort of spanning is part of that extension), so far as I can tell you could implement said extension for tiny hardware where every byte counts, and then for hardware where you're wanting to fetch 8+ instructions per cycle just ... not implement it?

Wait (he says to himself, realising he's an idiot immediately -before- posting the comment for once). You said upthread the C extension is specified as part of the standard UNIX profile, so I guess people are effectively required to implement it currently?

If that was changed, would that be sufficient to dissolve the issues for people wanting to design high performance implementations, or are there other problems inherent to the extension having been specified at all? (apologies for the 101 level questions, the only processor I really understood was the ARM2 so my curiosity vastly exceeds my knowledge here)

Have the ARM AArch64 designers ever commented on this? They intentionally left out any kind of compressed instructions, and certainly Apple at least cares a lot about code size.

Try this at 34:30 - from Arm’s architecture lead Richard Grisenthwaite. Earlier he says that several leading micro architects think that mixing 16 bit and 32 bit instructions (Thumb2) was the worst thing that Arm ever did.

https://m.soundcloud.com/university-of-cambridge/a-history-o...

He explicitly specifies that those micro-architects are at companies OTHER than ARM.

His own opinion appears to be that the worst thing ARM ever did was T2EE, designed for JIT compilers and compilers for dynamic languages. He says that by the time the chips came out compiler technology had advanced to the point that it was no longer useful and no one else used it.

A couple of other points picked up in the talk:

- He reverses Hennessy and Patterson wrt SPARC and MIPS.

- A64 effort started in 2007. So it took 5 years to freeze/publishing, the same as RISC-V.

- A64 architects thought code density is no longer important. Some people definitely disagree with that. At the time they probably thought amd64 was the only competition and matching/beating that was good enough.

- he seems to be regretting the 2nd operand shift because it fell naturally out of the 1985 micro-architecture, but it's a burden now. And yet it was included in A64 -- presumably because the initial processor pipelines had it anyway, because they supported A32. But now we have A64-only CPUs.

- LL/SC was the wrong thing to do.

Not only that, but CPUs have a maximum number of instructions they can dispatch per cycle (typically 4 or 6). Even in microbenchmarks, the difference there could show up.

The bottleneck on that is data interdependency of your algorithm. If you break it in 6 or 10 instructions, the data dependency stays the same.

(Of course, you can add unnecessary dependency with a badly designer ISA. But it's not a necessary condition.)

There's still a limit to how many instructions you can decode and dispatch every cycle, even with zero dependencies. There's also definitely dependencies in the example where you're computing a memory address to access a value.

Thank you for writing the obvious. Instruction Byte count is the wrong metric here 100%. Instruction Count (given reasonable decoding/timing constraints) is the thing to optimize for and indeed variable length encoding is very bad.

Instruction byte count matters quite a lot when you're buying ROM in volume. And today, the main commercial battleground for RISCV is in the microcontroller space where people care about these things.

For those of us without the expertise, could you elaborate on why that is?

On the one hand we have byte count, with its obvious effect on cache space used. But to those of us who don't know, why is instruction count so important?

There's macro-op fusion, which admittedly would burn transistors that could be used for other things. Could you elaborate why it's not sufficient?

And then the fact that modern x86 does the opposite to macro-op fusion, by actually splitting up CISC instructions into micro-ops. Why is it so bad if they were more micro-ops to start with, if Intel chooses to do this?