News for hackers. Stuff that matters.
Re:Why do CPUs have multiple cache levels? by exDM69 on Monday August 08, 2016 @05:13AM (#12246393)
There are about 180 registers in the register file already. There are only names for 16 or so registers, the CPU internally does renaming for them. It would be possible to specify an API that leaves a part of the named registers reserved for kernel, but that wouldn't really help.

When an interrupt or system call happens, all the registers get pushed to the stack. The stack is typically in L1 cache (and subject to various CPU optimizations), so it's really fast to push the 16*64 bit registers to the stack.

System calls, interrupts and context switches are "slow" not because they have trivial overhead like pushing registers. What's really consuming the time is secondary effects like TLB flushes, changes to the page tables, polluting the branch predictor, etc. It takes a very long time for the CPU to "warm up" again after a context switch.

Re:Why do CPUs have multiple cache levels? by xxs on Monday August 08, 2016 @05:55AM (#12246518)
System calls are mode switch that's way faster than a context (i.e. process) switch. The latter virtually always has cold caches.

I've made the remark because it's fairly common to mistake mode and context switches.

Re:Why do CPUs have multiple cache levels? by exDM69 on Monday August 08, 2016 @05:58AM (#12246528)
Yes, and the x86-64 has a fast SYSCALL instruction.
Re:Why do CPUs have multiple cache levels? by monocasa on Monday August 08, 2016 @12:27PM (#12248946)
Fast being in the thousands of cycles last time I checked.
Re:Why do CPUs have multiple cache levels? by the_mitsuhiko on Monday August 08, 2016 @02:55PM (#12250031)
The whole syscall maybe but certainly not the method of invoking it.
Re:Why do CPUs have multiple cache levels? by bogomipz on Monday August 08, 2016 @07:30AM (#12246878)
Can you elaborate on both there being 180 registers and the CPU does renaming of them? Surely the RAX register in an X86_64 CPU is always the same physical location no?
Re:Why do CPUs have multiple cache levels? by Tuna-Fish on Monday August 08, 2016 @10:09AM (#12247866)
> Surely the RAX register in an X86_64 CPU is always the same physical location no?

No, it is not.

Modern CPUs all use Physical Register Files or PRFs for implementing OoO. The way they work is that there is one backing register file of ~150-200 registers, and a naming table in the frontend. Each register file in the backing table can be in one of 3 states -- waiting for data, contains data, or clean. Every time an instruction that writes data to a register is executed, during the register rename stage the renamer picks one register from the clean set, assigns that as the output of the uop and the current value of the logical register, and sets it as waiting for data.

That is, let's say you execute:

add RAX, RAX

and the current value of RAX in the rename table was PRF#1, with PRF#1 carrying data and all other physical registers clean. In the renamer, this would then turn into:

add PRF#2, PRF#1, PRF#1

if this instruction was immediately followed by another add, RAX, RAX, it would then turn into:

add, PRF#3, PRF#2, PRF2, and after renaming that instruction RAX points to PRF#3.

and so on. In PRF machines, architectural registers only exist as pointers into the PRF.

The reason for this design is that it makes OoO execution simple, and reduces unnecessary data movement. An instruction is ready to execute once all it's inputs have data in the PRF, and when it executes there is no need to move data into some special architectural register.

Registers are reaped from the "haves data" set into the clean set once no instruction or architectural register points at them. Note that multiple architectural registers can point to the same physical register: mov rax, rbx is resolved in the frontend just by pointing rax to the same register as rbx, and there is typically a special register for the value 0 that all common register-clearing operations use.

Re:Why do CPUs have multiple cache levels? by bogomipz on Monday August 08, 2016 @03:34PM (#12250306)
This was a fantastic explanation so thanks. A couple of questions:

PRF is Physical register file? As opposed to the logical which can point to different backing store(the actual SRAM.)

Register renaming and OoO operations are really enabled by micro-ops? In other words on a CPU with completely hard-wired control unit such things would never be possible.

Re:Why do CPUs have multiple cache levels? by Tuna-Fish on Monday August 08, 2016 @07:20PM (#12251478)
> PRF is Physical register file? As opposed to the logical which can point to different backing store(the actual SRAM.)

Yep. PRF means the single backing store where all values go, and PRF systems are named for it because they are usually contrasted to the other very common way to implement OoO, ROB-backed systems, where there is a specific location for the final architectural value, and possibly multiple in-progress values in the ROB. Intel used values in the ROB in all their CPUs up to Sandy Bridge, which was their first PRF design.

(The main difference between the types is that ROB-backed is faster in circuit delay, but moves more data than a PRF design, so on the same process they potentially clock higher but produce more heat. PRF-based designs are also easier to scale wider than the ROB-based design.)

> Register renaming and OoO operations are really enabled by micro-ops? In other words on a CPU with completely hard-wired control unit such things would never be possible.

No, you can do renaming and OoO well without uops, it just requires that your instruction set is designed to be amenable to it. Before x86 had uops, it couldn't do OoO and the competing RISC CPUs were way faster because of this. PPro added uops precisely because they made it possible for an x86 cpu to do OoO, and eventually x86 beat all the competing RISC chips at their own game.

Re:Why do CPUs have multiple cache levels? by monocasa on Monday August 08, 2016 @05:35PM (#12251041)
> PRF is Physical register file?

Yep, it's the actual renamed registers.

> Register renaming and OoO operations are really enabled by micro-ops? In other words on a CPU with completely hard-wired control unit such things would never be possible.

No, they're orthogonal concepts. The first OoO processor (the System 360 model 91) didn't have uops.

Re:Why do CPUs have multiple cache levels? by bogomipz on Monday August 08, 2016 @07:00PM (#12251396)
Sorry that shouldn't have said OoO, register renaming and micro-ops I meant.
Re:Why do CPUs have multiple cache levels? by monocasa on Monday August 08, 2016 @07:21PM (#12251480)
It did register renaming too. It was designed by Tomasulo of Tomasulo's algorithm.
Re:Why do CPUs have multiple cache levels? by bogomipz on Monday August 08, 2016 @08:48PM (#12251830)
Wow, this is kind of blowing my mind that they were doing this with a hard-wired control unit in 1969(I think I have the date right.) Big Blue, its easy now a days to forget just how amazing they were. Do you have links you can recommend? Thanks.
Re:Why do CPUs have multiple cache levels? by JoeAltmaier on Monday August 08, 2016 @10:12AM (#12247881)
And the 'Mill' design just uses a single pointer into a ring buffer? Cool.
Re:Why do CPUs have multiple cache levels? by Tuna-Fish on Monday August 08, 2016 @10:22AM (#12247970)
Actually, the Mill uses just the forwarding network which I completely ignored in this explanation.

The really short explanation about the forwarding network is that writing/reading the PRF takes too much time for you to read it and do work on the same cycle. If you operated on it alone, all operations that were dependent on each other would have multi-cycle latencies.

Since this sucks, in addition to the PRF the machines have a forwarding network, where the result of every instruction is broadcast to all the execution units of a similar type. If the broadcasted PRF number matches the register you want, it's read from the forwarding network instead of the PRF.

AFAICT, the Mill works by completely eschewing the final register file, and exclusively using the forwarding network with some local storage at each execution unit for the past values on the network.

Re:Why do CPUs have multiple cache levels? by JoeAltmaier on Monday August 08, 2016 @10:24AM (#12247990)
Excellent description! Thanks.
Re:Why do CPUs have multiple cache levels? by ikeyany on Monday August 08, 2016 @08:05AM (#12247037)
I am not who you responded to, but: the registers are always in the same physical location, but internally, they are virtually renamed for efficient reuse and to avoid resource contention. If the CPU knows the stream of assembly instructions to be executed, it can abstract away actual register names and show a larger number of registers than what's physically available.
Re:Why do CPUs have multiple cache levels? by Symmetry on Monday August 08, 2016 @11:26AM (#12248502)
A larger cost would be from the extra bits in the instruction stream it takes to specify the new registers and the resultant extra icache pressure.
Re:Why do CPUs have multiple cache levels? by joseraul on Monday August 08, 2016 @07:05AM (#12246766)

  it's really fast to push the 16*64 bit registers to the stack
Since the CPU has 180 registers (with only 16 names), why don't we need to push all 180 to store context?
Re:Why do CPUs have multiple cache levels? by Dylan16807 on Monday August 08, 2016 @07:41AM (#12246921)
All that extra state is there to help overlap instructions on a time scale of nanoseconds. Every instruction has its own name->register mapping. If you're switching to kernel mode or maybe if you miss a branch or for whatever other reason, the CPU consolidates you back down to running one instruction. Once that happens, you only have one name->register mapping and all the registers have their correct values. The hidden state is reduced to nothing and you only have to save the "real" state of those 16.
Re:Why do CPUs have multiple cache levels? by cfallin on Monday August 08, 2016 @11:24AM (#12248475)
To expand on another answer here -- there are more values in the PRF than there are names because "old" instructions in flight can refer to outdated versions of architectural registers (the names). But a new instruction can only see the latest versions of the architectural registers, so only those 16 actually matter w.r.t. future execution: when we switch back to the task, its new instructions will ony need the 16 live values.
Re:Why do CPUs have multiple cache levels? by exDM69 on Monday August 08, 2016 @07:08AM (#12246785)
Because the CPU works "as if" there were only 16.
Re:Why do CPUs have multiple cache levels? by bogomipz on Monday August 08, 2016 @03:22PM (#12250235)
But modern CPU TLBs even have context process IDs(CPIDs)now so many times the whole cache doesn't totally need to "warm up" again.