on x64, token threading is
; goto tokens[program[ip]]
mov r0, [program_instructions + ip]
jmp [tokens + r0]
while direct threading is ; goto program[ip]
jmp [program_instructions + ip]
the paper saysNehalem shows a few outstanding speedups (in the 30 %– 40 % range), as well as Sandy Bridge to a lesser extent, but the average speedups (geomean of individual speedups) for Nehalem, Sandy Bridge, and Haswell are respectively 10.1 %, 4.2 %, and 2.8 % with a few outstanding values for each microarchitecture. The benefits of threaded code decreases with each new generation of microarchitecture.
but looking at the graphic, it seems a few benchmarks still had a +5% speed boost on Haswell.
It would also have been interesting if the paper provided the generated code by gcc for the switch statement. If the switch statement's cases are linear (0, 1, 2, 3, ...) and if there are more than 4 cases, then gcc token threads the code.[2]
[1] https://hg.python.org/cpython/file/v3.3.2/Python/ceval.c#l82... [2] https://godbolt.org/g/yRjFto
https://news.ycombinator.com/item?id=16777516
that is, "system" VMs vs. "process" VMs. I don't really like those terms, but the distinction is a good one. I think the author must have been at the VM summer school I linked there.
-----
BTW this 2015 paper is saying that the labeled gotos technique isn't a big deal on Haswell:
Branch prediction and the performance of interpreters -- Don't trust folklore
https://scholar.google.com/scholar?cluster=20257610669850946...
In other words the branch predictors improved enough on Intel such that it's not a big optimization. But I don't like how they didn't mention other CPU architectures. Intel is not all we care about!
Also, it would be nice to have an update on this 2015 paper to account for 2017-2018 Spectre/Meltdown mitigations...