Note macro op fusion is widely used for other architectures already, particularly ones like x86 where what the processor actually runs looks nothing like the machine code.
It doesn’t matter if they’re fused or not if the reduced instruction density increases memory usage and puts more pressure on I$.
Also, I don’t buy the whole fusion argument on the grounds that having to fuse super complex (5 instruction or more) sequences adds enough complexity that you’ve got opportunity cost. Much better for everyone if the CPU doesn’t have to do that fusion. That’s the whole point of good ISA design - to prevent the need for fusing in cases you’re doing something super common.
Edit: Here's the thesis about the design decisions in the C encoding: https://people.eecs.berkeley.edu/~krste/papers/waterman-ms.p... See also the diagram on page 62 of this document: https://people.eecs.berkeley.edu/~krste/papers/EECS-2016-1.p...
The thing I see is this: you can also add compressed encodings for any ISA. And that has its own costs (it’s harder to decode and it’s harder on software that wants to do bidirectional analysis of machine code). So “my isa has shortcomings but it’s cool because compression” isn’t a perfect argument since if your isa lacks those shortcomings then you still benefit from compression and you don’t need it as much, which is better.
not necessarily, since
1. the compressed versions are basically 1-1 to the non-compressed versions
2. the uncompressed versions are more for study/academia and clarity; its expected that irl only compressed instructions are used (for instructions that can be compressed)
more info: https://riscv.org/wp-content/uploads/2015/11/riscv-compresse...
(IIRC they removed that one on x86-64)
RISC-V compact instructions don't require special modes and run in fully-mixed mode with 32-bit instructions without all the penalties thumb has (they are literally just extended into their 32-bit counterparts internally).
High code density became less valuable, was the point.
Can you give an example of someone advocating for 5 instruction fusion? Normally it's limited to three.
add t0, t1, t2
slti t3, t2, 0
slt t4, t0, t1
bne t3, t4, overflow
edit: so I now think a good extension to add overflow checking to RISC-V is with an instruction that works like "slt"- call it "sov", set if add would overflow:
add t0, t1, t2
sov t3, t1, t2
bnez t3, overflow
The only way to know there isn't a dependency is if that register gets clobbered by something else very soon afterwards.
But this whole topic of "checking for signed overflow is expensive" is overblown. It's simply not that important an operation, especially in the context of those languages that do it a lot also doing a lot of memory references, which are far more expensive.
Adding arbitrary completely unknown integers is pretty rare. If you know both numbers are greater than zero then a single compare-and-branch is all you need. If one of the numbers is a constant then a single compare-and-branch is all you need.
auipc lr, zero, .LONG_TARGET20
jalr lr, lr, .LONG_TARGET12
So in parent's example, sov might fuse with the following bnez, but it likely wouldn't fuse with the preceding addition.
Quite a few things determine whether a fusion is doable or not. In addition to the number of destination registers, you do, to a more relaxed extent, care about source operands, but also things like ‘does this fit nicely in a single pass through the pipeline?’ and even just ‘is this materially beneficial?’
Lots of cores (but not all) can write two registers from a fused instruction, given the right conditions, and sov does rerun the addition, so add-sov fusion sounds very doable to me.
Bitfield insertion is only one instruction in most RISC ISAs, but 5 or more in RISC-V.
That said, until it gets ratified by the consortium and implemented in silicon its still just a (well-researched) wishlist.
You need fast sequences for all of these variants:
- add or sub or mul
- signed or unsigned
- 32 bit or 64 bit
Some of those need 5 instructions. I don’t remember which adventure you need to pick to get 5.
sadd32, sadd64, uadd32, uadd64, ssub32, ssub64, usub32, usub64, smul32, smul64, umul32, umul64
I don't think it's a fusion problem; even if you did fuse these sequences, they'd still be bad, since they'd be writing lots of extra registers.
And worth noting that these are just the ones where you usually have good sequences on other cpus, but even then (like signed mul) they’re not perfect. Lots of room for improvement. It’s just that risc-v didn’t seize the opportunity.
(Quoted from OPs comment)
This isn’t a subject I’m an expert at but wouldn’t this mean there’s already some sort of translation going on already on the other systems? So it’s mostly just added end user work not a giant performance loss on RISC?
It would just then simply be a layer of abstraction that is lost.
Can't wait to see more information about the goldmont microcode work to see if that holds for intel as well as it does for AMD.
The x86 front end breaks down big instructions into smaller RISC-like micro-ops and then fuses/re-orders/optimizes/etc and runs those instead. There’s pros and cons, the con being sheer complexity and power budget, the pros are that it’s an abstraction so the microarchitecture can change completely without recompiling your code —- and you get CPU specific optimizations too. The CPU is basically emulating x86.
You could in theory build an x86 CPU with a RISC-V core behind that decoder.
This is a widely held meme, but the internet at large doesn't have any evidence to back it up. A couple of publicly visible engineers that do have experience are on record as saying that cell-phone-class competitive x86 was absolutely possible. Intel and AMD chose not to pursue those markets.
The expensive parts of a high-end CPU aren't normally in the instruction decode part. They are in the branch prediction, branch mispredict recovery, forwarding networks, memory re-ordering, and so on. Anything short of a dataflow ISA has little impact on those structures.
Don't think its strictly accurate to say that Intel 'chose not to pursue' the mobile SoC market - IIRC they tried, made little progress and gave up having spent a lot of money in the process.
[1] https://www.usenix.org/system/files/conference/cooldc16/cool...
I don't think you can generalize from that result to much of anything.
That struck me as one of the few apples to apples comparisons ever of the instruction sets at the high end, from a party not really incentivized to bend the truth one way or another.
But it could have easily come from something like the relaxed memory model, or they could have just been overly optimistic. The chip was cut after all.
With the exception of Atom, all recent Intel designs have been a RISC core with a CISC decoder slapped on top. Everything else being equal, the simpler decoder will create a smaller chip. Because the decoder is always running all-out, the simpler decoder will also use less power.
x86 instructions are multi-length from 1 to 15 bytes. The cost to slice that up is always going to be bigger than fixed-length instructions. RISC-V has variable length in theory, but in practice, compact instructions extend into 32-bit instructions with some bits added which is important for decode cache while longer instructions are ignored.
Because there's a maximum tolerable decode latency of a very few cycles and latency increases with cache size, decode cache size has a definite cap. x86 has a couple orders of magnitude more potential instructions than RISC-V. More instructions translates into a lower hit rate for the same size cache barring any heuristics (more on that below).
Matching variable-length arrays to an unknown set of arrays in cache is inherently a hard problem. Every solution has tradeoffs and the resulting heuristics are bad for computing (see below). In contrast, searching for a match on a fixed 32-bit array has much more simple general solutions that don't require tradeoffs.
C and CPU designs feed off each other. Let's say there are instructions X and Y which can do equivalent things. x86 engineers played around with both and got a lucky insight into how to make X a bit faster. Compiler writers jump on it and start using the faster solution. x86 engineers now all but stop looking to improve Y and spend their time tinkering with X instead. Compilers now focus even more heavily around not just X, but any instructions more closely associated with X.
In that entire (true) story, nobody gave a second thought to whether the final result of Y would have been faster overall if not for the lucky break with X. If x86 developers were actually free to choose whichever instructions they wanted, x86 decode would be much, much slower than it appears to be. This self limitation argues that perhaps a more RISC-like ISA is inevitable.
A new ISA where everything is used would definitely have complexity, transistor, and power disadvantages vs a new ISA that didn't make that mistake.
Weren't there Windows Phone devices with x86 SoC, but they weren't competitive?
Commercially made x86 Android phones exist, the most popular to my knowledge were some of the Asus ZenPhone models.
When Atom was released in 2008, the A9 had already been announced (a year before). A9 was around 10-15% faster per clock and were often multicore meaning a 1.5GHz chip was faster in all metrics over the 1.6GHz Atom.
A few articles came out June/July of this year with Analysts saying that Intel had spent over 10 Billion dollars trying to break into the mobile market with no success. ARM's current R&D budget (according to Nvidia a month or so ago) is 0.5B. If they spent that much every year since 2000, they would barely match Intel, but their entire market cap in stayed under 4B all the way until 2009. Remember, that R&D includes their high-end ARM cores, but also GPU, midrange designs, various microcontroller designs, a realtime OS, ARM tooling, NPUs, various kernel support, etc
If that much money can't fix up x86 to keep up with a budget a fraction of the size, I take that as proof that the ISA really does matter.
// &(array[offset])
slli rd, rs1, {1,2,3}
add rd, rd, rs2
As an aside, LEA is never getting added to the base RISC-V nor should it be. But I'm surprised it isn't considered for an extension.
[1] https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-...
I'm not sure that is the idea, given that RISC-V is targeted at processors so low-end that they don't even implement multiply.
RISC-V has some closely-related sharp corners in indexed address arithmetic as well. Some choices for the type of the index variable perform much worse on rv64.
Consider: an LP64 machine uses 32-bit integers for 'int' and 'unsigned', but 64-bit integers for `long`, `size_t`, `ptrdiff_t` and so on.
If you use an array index variable of type `unsigned`, then the compiler must prove that wraparound doesn't happen. That's pretty weird considering that half the point of using unsigned is to elide such proofs of correctness. If it cannot prove the absence of unsigned wraparound, then it will be forced to emit zero-extension sequences prior to using the index variable to generate the addresses.
ARMv8 side-steps the whole problem by providing indexed memory addressing modes that include the complete suite of zero and sign extension of a narrow-width index in the load or store instruction itself.
So here we have an example of a three-way system engineering choice.
- Provide a small amount of hardware that performs the operation on-demand.
- Provide new and inventive forms of value-range analysis in the compiler. Despite decades of research into this problem, the world's best solutions still frequently saturate at "the entire width of the type the programmer requested".
- Change the habits of the world's C programmers.
This is usually why your array indexing should be done with an iterator or size_t :)
The problem isn't with unsigned types generally. Its with subregister unsigned types. So, size_t and uintptr_t are fine. uint32_t, uint16_t, uint8_t (on LP64 ABIs) are pessimized and demand zero-extension instructions (or proofs that they can be safely elided) prior to causing side-effects. uint64_t on a LP128 ABI would also be problematic.
signed 32-bit int is also fine... because RISC-V specifically has a suite of arithmetic instructions that unconditionally sign-extend from bit 31. Even without those, it would still be fine because the carve-out for undefined behavior is wide enough for INT_MAX+1 to remain positive. Same thing for all of the other narrow-width signed integer types. If you increment SHRT_MAX and then use it to index memory, its perfectly legal undefined behavior to access base + SHRT_MAX+1 instead of base + SHRT_MIN.
However, that's not legal for the unsigned types. They are all mandated to wrap in 2's complement. base + UINT_MAX+1 must access base + 0 when the index is `unsigned int`, even on a 64-bit machine.
Ironically, given the topic, that's actually not true, because `base + UINT_MAX+1` is `(base + UINT_MAX)+1` (with a pointer, not a unsigned int, as the temporary value). That should probably be `base + (UINT_MAX+1)`.
But isn't size_t (or ptrdiff_t) the preferred indexing type in C for this reason (among others)? Sometimes you of course do want wrap around modulo semantics but that's much rarer, right?
How much business code out there burns most of its cycles in Java/C#/Rust/Go? Having even one of those (lets face it: Java) would have gone a very long way. How much client computing spends most of its cycles in (JITted) javascript?
I understand the tradeoff that drives this omission, though. How many VC megabucks (Tens? Hundreds?) would they have had to spend commercially on a bet? Alternatively, how many grad student-years grinding through low payoff work to prove yea/nea on the value of checked arithmetic?
Lets say that you make a hip-shot bet on some checked arithmetic support without the supporting toolchains and application code to back up the design choice. You could end up making a different set of mistakes instead and end up in the same situation.
I think the only mistake was in finalizing the ISA without any support for checked arithmetic. My belief is that doing it well will not be orthogonal to the rest of the ISA's design, and therefore is a poor candidate for an extension.
ISO C is therefore a three-way compromise between three communities: Software authors, compiler authors, and CPU manufacturers. You will naturally end up with some decisions that dissatisfy some members of each.
1) it’s pretty important to have easy syntax for math with overflow checks. Otherwise there will be bugs. Bad bugs. We have those and it sucks.
2) it’s pretty important that if the overflow wraps in ISA then it wraps in language semantics but C just says “meh whatever” for signed overflow. That leads to even more bugs.
GCC has intrinsics for integer math with overflow checks
I don’t think these folks have seen what modern languages do.
Overflow is part of the result, so maybe include extra bits to each register that can be arithmetic destination. These bits are not included in moves, but could be tested with new instructions.
Another way that avoids flags is new arithmetic instructions: add but jump on overflow. Maybe this is reduced to add and skip next instruction except for overflow, but maybe things are simplified if the only allowed next instruction is a jump, so the result is a single longer instruction.
add t2, t1, t0
sov t3, t1, t0
bnez t3, overflow
"add but jump on overflow" or "add and skip on no overflow"- I don't like this because you can not break it into separate operations without flags. I think you might have to add hidden flags in a real implementation.
An add followed by an sov could be fused, but requires an expensive multi-register write. Fusing maybe could be more likely if the destination is always to a fixed destination register:
add t2, t1, t0
sov tflags, t1, t0
bnez tflags, overflowaddsov t2, t1, t0
The best design is part of the core ISA and not and extension since overflow checking is fundamental to modern languages.
"Several ISA features, including implicit condition codes and predicated moves, are onerous to implement in aggressive microarchitectures. Yet, their complexity often does not result in higher performance because their semantics were ill-conceived. For example, x86 provides a conditional load instruction, but, if the unconditional load were to cause an exception, it is implementation-defined whether the conditional version would do so. Thus, a compiler can only rarely use this instruction to perform the if-conversion optimization.
Recognizing the inefficiency of their conditional operations, Intel’s recent implementations go to some lengths to fuse comparison instructions and branch instructions into internal compare-and-branch operations."
Yeah, doing condition codes right is complicated but then that's the purpose of microarchitecture, to factor that complexity out.
RISC-V succeeds in having a minimalist to a fault design which is appropriate for a certain design point. ARMv8 and x86_64 are more useful for a broader set of designs. The burden now is on RISC-V to show that their minimalist approach is fast+efficient rather than just simple. You have to get something out of the simplicity; otherwise what you get is design debt.
I like the Mill CPU approach, where every "register" (it doesn't have named registers actually) has the full set of status bits associated with it, and not just for overflow. Things like "not a result" (NaR), which can represent the result of a failed speculative load for example (because the process doesn't have permission to read from that page, for example).
The status bits part in general, or the speculative load stuff?
They allegedly have all this working, privately. They haven't released any development tools or such to the public.
I've often toyed with the idea of writing an instruction-level simulator (as opposed to the RTL sim or whatever they have internally). But even sticking to the public information, I'd likely be infringing on their patents.
Basically, if there's a bottleneck to x86 code, Intel has run into it, profiled for it and generally optimized around it both in their microarchitectures and in the their C compiler.
If you did overflow checks rarely then what you say is a very good point indeed. The key thing is just the frequency of this stuff in modern languages.
If you doubt that dynamic languages are using overflow checking in the way that I describe then it’s not because of lack of papers on the subject.
Off-topic, but could you point me to more details on this? Someone (else?) recently mentioned octagon analysis in JSC in a HN thread. I grepped through the sources at the time but didn't find any indication that it exists. At least not under the name "octagon".
Look at DFGIntegerRangeOptimizationPhase.cpp
I think an important design point here is that the languages that need a lot of dynamic overflow checks are primarily used on beefier CPUs so if you can get around the code size issue, making it performant only on more capable designs is fine since the overflow check will be rare on simpler CPUs.
1. Folks totally run JS and other crazy on small CPUs.
2. Other safe languages (rust and swift I think?) also use overflow checks. It’s probably a good thing if those languages get used more on small cpus.
3. The C code that normally runs on small cpus is hella vulnerable today and probably for a long time to come. Compiling with sanitizer flags that turn on overflow checks is a valuable (and oft requested) mitigation. So theres a future where most arithmetic is checked on all cpus and with all languages.
And yeah, it’s true that the overflow check is well predicted. And yeah, it’s true that what arm and x86 do here isn’t the best thing ever, just better than risc-v.
But to a first approximation, if you double the density of conditional branches in the program, then you will need to roughly double the size of the branch prediction tables to get the same performance, even if all of them are correctly predicted 100% of the time.
Currently only the most basic extensions are available. But nothing prevents RISC-V from introducing in the future an extension that extends the conditional code or an extension for integer/float overflow.
RISC-V means three instructions instead of two in the best case. It requires five or more instead of two in bad cases. That’s extremely annoying since these code sequences will be emitted frequently if that’s how all math in the language works.
Also worth noting, since this always comes up, that these things are super hard for a compiler to optimize away. JSC tries very aggressively but only succeeds a minority of the time (we have a backwards abstract interpreter based on how values are used, a forward interpreter that uses a simplified octagon domain to prove integer ranges, and a bunch of other things - and it’s not enough). So, even with very aggressive compilers you will often emit sequences to check overflow. It’s ideal if this is just a branch on a flag because this maximizes density. Density is especially important in JITs; more density means not just better perf but better memory usage since JITed instructions use dirty memory.