As far as I can tell the reason is that with variable length instructions you have a mux layer between fetch and decode that sends the right set of bytes to the right decode slot. Your first set of 16 bits is certainly going into the first decoder but your fourth might be going to the second, third, or fourth depending on previous decoding. Or maybe not, I've only worked with fixed width instructions when getting into the weeds of processors and only entirely designed a simple five stage classic RISC myself. So don't take the above too seriously.

What you should take seriously is that three of the four big RISC ISAs - MIPS, ARM, and POWER - all have extensions for variable length instructions. They all use them in embedded systems where program memory size is a big factor in your bill of materials and reducing that can save you money. But none of them use it on their high end processors. So I think we can conclude that the power/transistor cost of fetching a number of variable length instructions per clock is higher than the cost of expanding the L1 instruction cache to deal with full size instructions.

Thumb works as an alternative encoding that you have to switch to and where you just have a subset of the instructions available. Switching has at least the same cost as a branch. My guess is that this is the reason why thumb is avoided often.

RISC-V on the other hand has an encoding scheme that already from the beginning allowed instructions with longer and shorter encodings. You can just mix compressed instructions with normal ones.

Thank you for clarifying. It still seems wrong, somehow. Admittedly I don't know enough detail about Thumb and the others, but if the variable-length encoding is a simple design like RISC-V's, it's hard to imagine that extra layer to be so expensive as to eliminate the benefits of an effectively increased I$. It would be nice to be able to dig deeper into the details of this.

At most, you're looking at an extra pipeline stage for all of that. The less I$ pressure is almost certainly worth it.

The real cost you're looking at is the complexity for instructions to straddle cache lines and MMU pages, but even then it's less of a deal than you might think.

Nope, done right the cost is a layer of 2:1 muxes and some control logic to pass data between decoders, and 16 flops to hold a partially decoded instruction between fetch bundles

It's more complicated than that if you want to decode multiple instructions per cycles, which you have to for high-end designs. I still agree with monocasa that it's almost certainly going to be worth it for the reduced I$ pressure.

For sure, it's just that this little bit could necessitate a pipeline stage if that was the critical timing path somehow already. Just trying to be as friendly to their argument as possible before taking it on.

x64 has that problem, in the extreme (max length of an instruction is 15 bytes) doesn’t it?

It must be a pain to design its instruction decoder, but I don’t think that’s insurmountable for high end processors.

Or do I misunderstand your argument?

Yes, it certainly does. As far as I can tell it takes more engineer hours to make an efficient x86 front end than it does a RISC front end and the RISC designs I see tend to balance their pipelines so that they're wider in front than x86 designs typically are. Everything here is a matter of "all else being equal" and the talent and hard work of the design team are more important than any of the ISA differences we're talking about here.