Not being able to fetch new cachelines when under speculation would be a huge blow, as exposing memory level parallelism is one of the most important features of OoO CPUs.
Edit: also fetching a cacheline is not really undoable as it is observable from other CPUs via the coherency protocol: i.e. while it might be possible to hide to the local cpu that a cacheline was loaded under a failed speculation, the effect can be still observed by another core by noticing that an exclusive line is now shared (by timing for example the latency of an atomic instruction)
Yes, it is. Remember, again, that the vast majority of those instructions can still be speculated, because the vast majority of instructions are not loads or stores.
Now, certainly, the expensive ones are loads and stores, but i'm just pointing out that the hundreds of instructions you are talking about in the buffer are mostly not loads and stores.
It's true that lowering memory level parallelism would be a huge blow, as the vast majority of time in well-tuned cpu bound apps is usually spent in stalls waiting for memory (otherwise, if it's really just arithmetic bound, it may make more sense to run it on a GPU or something), and this would just increase it.
The real question is what percent can you prove are safe to speculate, and at what point can you prove that safety (ie assuming it must be dynamically speculated, can you prove safety with enough cycles left that it matters). If you have 5 instructions, yeah, no, probably not.. But it may also be the case that the execution environment can prove it safe for you as the program executes and tell you.
I expect getting back this performance is going to be done using a variety of methods, some cooperation between jits/compilers and processors, and possibly some weird abstractions around marking memory you want to protect or not (IE not speculate around).
I mean, in the absolute worst case, you could make loads/stores take constant time and speculate as much as you like :)
It's just that this has a much higher performance cost right now than not speculating at all (by a few orders of magnitude)
I think realistically the only safe and not completely performance crippling workaround will be, at the very least, to run any untrusted code under a separate address space (assuming that the cpu is immune from meltdown). That doesn't necessarily require a full blown separate process, but something like memory protection keys might work.
The alternative is full static analysis and source level annotations, which realistically is only going to be done for very few programs and will still be error prone.
Otherwise, they can observe it before you roll it back due to the way the coherency protocols work.
> Compilers will get called upon to do more safe speculation
Compilers can also emit code that minimizes those situations. Also, extending the ISA with an instruction modifier that signals that otherwise innocent instruction has indeed observable side effects (ideally the processor should know that, but, if the compiler already knows it, a runtime check can be skipped.
> Processors will have to grow the logic to determine when speculation is safe
In many cases, this would be as simple as flagging the micro-op as unsafe and pause the speculation at that point.
For Meltdown, the just-add-silicon approach could be to never share cache between privileged and unprivileged code. To extend that to Spectre, never share cache across different PIDs (but, then, the ISA will have to know what a PID is). Since that would reduce the effective cache effectiveness, caches will have to grow.
Fun times ahead.
Let's ignore meltdown, which seems solvable in hardware with no obvious performance loss (assuming amd's existence proof is correct), and concentrate on spectre, which everyone thinks means the world is ending.
One thing to note is that compilers have been safely speculating instructions for years. Shocking, i know :) Processors could too.
They just weren't. One of the cardinal rules of safe speculation is that you can't speculate an instruction unless you can prove it has no observable side-effects (ie possibly faulting, or in this case, ending up in cache), or that the side-effects are the precisely the same whether you speculate it or not (where "precisely the same" generally includes some notion of ordering of side-effect occurrence. i'm not going to get into all the various intricacies)
IE
Compilers generally did not speculate loads, mainly because they can fault at that level[1]. The fault comes from the processor though.So processors were/are assuming that speculation of loads, calls, etc, had no side effects because they could "throw away the results". As the processor controlled the faulting, it could just throw away the fault and pretend it never happened.
These two attacks are just proving that is not true, and the side-effects of computation themselves are observable.
The end result should be the same. Processors speculate more like compilers do: only in safe situations.
The idea that you have to give up all speculative execution seems very wrong.
You only have to give it up in cases where there are possibly observable side-effects and it's not guaranteed they will always happen.
The upshot is the most likely outcome is that both processors and compilers will work harder to speculate.
Compilers will get called upon to do more safe speculation.
Processors will have to grow the logic to determine when speculation is safe (or figure out a way to actually undo all side effects, which is fairly hard).
Now, the downside is the most useful speculation is obviously to hide load/store latency, and those things are the hardest to reason about safety.
But like i said, compilers have been doing it for many years at this point. Our hardware brethren just found out the hard way that they are likely to start having to do it too, and that there are more observable side-effects than they thought.
[1] Some JITs do it and catch the result in a fault handler if it turns out badly. I expect someone is going to discover ways to exploit all of these basically instantly. These will not be fixed by processor related fixes because they are not processor directed.