Have we learned nothing from Spectre and Meltdown?... :(

The problem is that we're more or less stuck with this class of problem unless we end up with something that looks like a Xeon Phi without shared resources and run calculations on many, many truly independent cores, or we accept that the worst and best case performance cases are identical (which I don't foresee anyone really agreeing to).

Or, framed differently, if Intel or AMD announced a new gamer CPU tomorrow that was 3x faster in most games but utterly unsafe against all Meltdown/Spectre-class vulns, how fast do you think they'd sell out?

Larabee was fun to program, but I think it'd have an even worse time hardening memory sideband effects: the barrel processor (which was necessary to have anything like reasonable performance) was humorously easy to use for cross-process exfiltration. Like... it was so easy, we actually used it as an IPC mechanism.

> it was so easy, we actually used it as an IPC mechanism.

Can you elaborate on that? It sounds interesting

Now you’re asking me technical details from more than a decade ago. My recollection is that you could map one of the caches between cores — there were uncached-write-through instructions. By reverse engineering the cache’s hash, you could write to a specific cache-line; the uc-write would push it up into the correct line and the “other core” could snoop that line from its side with a lazy read-and-clear. The whole thing was janky-AF, but way the hell faster than sending a message around the ring. (My recollection was that the three interlocking rings could make the longest-range message take hundreds of cycles.)

Sure, absolutely, there's large numbers of additional classes of side effects you would need to harden against if you wanted to eliminate everything, I was mostly thinking specifically of something with an enormous number of cores without the 4-way SMT as a high-level description.

I was always morbidly curious about programming those, but never to the point of actually buying one, and I always had more things to do in the day than time in past life when we had a few of the cards in my office.

We already have heterogeneous cores these days, with E and P, and we have a ton of them as they take little space on the die relative to cache. The solution, it seems to me, is to have most cores go brrrrrr and a few that are secure.

Given that we have effectively two browser platforms (Chromium and Firefox) and two operating systems to contend with (Linux and Windows), it seems entirely tractable to get the security sensitive threads scheduled to the "S cores".

The main security boundary a modern computer upholds is web vs everything else, including protecting one webpage from another.

So I think it should be the javascript that should run on these hypothetical cores.

Though perhaps a few other operations might choose to use them as well.

Also all the TLS, SSH, Wireguard and other encryption, anything with long-persisted secret information. Everything else, even secret (like displayed OTP codes) is likely too fleeting for a snooping attack to be able to find and exfiltrate it, even if an exfiltration channel remains. Until a better exfiltration method is found, of course :-(

I think we're headed towards the future of many highly insulated computing nodes that share little if anything. Maybe they'd have a faster way to communicate, e.g. by remapping fast cache-like memory between cores, but that memory would never be uncontrollably shared the way cache lines are now.

That's a secure enclave aka secure element aka TPM. Once you start wanting security you usually think up enough other features (voltage glitching prevention, memory encryption) that it's worth moving it off the CPU.

That's a wildly different type of security. I just want to sandbox some code, not treat the entire world as hostile.

Eh, the TPM is a hell of a lot less functional than security processor on a modem arm board. You can seal and unseal based on system state, but once things are unsealed, it's just in memory

"if Intel or AMD announced a new gamer CPU tomorrow that was 3x faster in most games but utterly unsafe against all Meltdown/Spectre-class vulns, how fast do you think they'd sell out"

Well, many people have gaming computers, they won't use for anything serious. So I would also buy it. And in restricted gaming consoles, I suppose the risk is not too high?

Consoles are hardened very well to prevent homebrew, cracking, cheating etc.

But this class of vuln is about data leaking between users in a multi-user system.

Isn't it rather about data leaks between any two processes? Whether those two processes belong to different users is a detail of the threat model and the OS's security model. In a console it could well be about data leaks between a game with code-injection vulnerability and the OS or DRM system.

They're a pain in the ass all around. Spectre allowed you to read everything paged in (including kernel memory) from JS in the browser.

To mitigate it browsers did a bunch of hacks, including nerfing precision on all timer APIs and disabling shared memory, because you need an accurate timer for the exploit - to this day performance.now() rounds to 1MS on firefox and 0.1MS on Chrome.

This 1MS rounding funnily is a headache for me right as we speak. On a say 240Hz monitor, for video games you need to render a frame every ~4.16ms -- 1ms precision is not enough for accurate ticker -- even if you render your frames on time, the result can't be perfectly smooth as the browser doesn't give an accurate enough timer by which to advance your physics every frame.

you mean those consoles that can attack the rest of your devices and your neighbours via its wireless chips?

Speculation attacks enables code running on the machine to access data it shouldn't. I don't see how that relates to your scenario.

Also, many games today outright install rootkits to monitor your memory (see [1]) - some heartbleed is so far down the line of credible threats on a gaming machine that its outright ludicrous to trade off performance for it.

[1]:https://www.club386.com/assassins-creed-shadows-drm-wants-to...

> how fast do you think they'd sell out?

10-20 min, depending on how many they make :)

Also, a good chunk of these vulnerabilities (Retbleed, Downfall, Rowhammer, there's probably a few I'm forgetting) are either theoretical, lab-only or spear exploits that require a lot of setup. And then the leaking info from something like Retbleed mostly applies to shared machines like in cloud infrastructure.

Which makes it kind of terrible that the kernel has these mitigations turned on by default, stealing somewhere in the neighborhood of 20-60% of performance on older gen hardware, just because the kernel has to roll with "one size fits all" defaults.

If you know what you're doing, you do something like this: https://gist.github.com/jfeilbach/f06bb8408626383a083f68276f... and make Linux fast again (c).

If you don't know what kernel parameters are and what do they affect, it's likely safer to go with all the mitigations enabled by default :-|

Yeah, I personally have Retbleed and Downfall mitigations disabled, the rest thankfully doesn’t severely affect my CPU performance.

Appreciate sharing the gist though!

I don’t think you are thinking of this right. One bit of leakage makes it half as hard to break encryption via brute force. It’s a serious problem. The defaults are justified.

I think things will only shift once we have systems they ship with fully sandboxes that are minimally optimized and fully isolated. Until then we are forced to assume the worst.

> I don’t think you are thinking of this right. One bit of leakage makes it half as hard to break encryption via brute force.

The problem is that you need to execute on the system, then need to know which application you’re targeting, then figure out the timings, and even then you’re not certain you are getting the bits you want.

Enabling mitigations For servers? Sure. Cloud servers? Definitely. High profile targets? Go for it.

The current defaults are like foisting iOS its “Lockdown Mode” on all users by default and then expecting them to figure out how to turn it off, except you have to do it by connecting it to your Mac/PC and punching in a bunch of terminal commands.

Then again, almost all kernel settings are server-optimal (and even then, 90s server optimal). There should honestly should be some serious effort to modernize the defaults for reasonably modern servers, and then also have a separate kernel for desktops (akin to CachyOS, just more upstream).

Maybe so but I think most users are going to be vulnerable to likely under-estimate their security-sensitivity than to over-estimate. On top of that security profiles can change and perhaps people won’t remember to update their settings to meet their current security needs.

These defaults are needed and if the loss is so massive we should be willing to embrace less programmable but more secure options.

Itanium allegedly was free from branch prediction issues but I suspect cache behavior still might have been an issue. Unfortunately it's also dead as a doornail.

I agree at a gut / instinct level with that thought.

SINGLE thread best and worst case have to be the same to avoid speculation...

However for threads from completely unrelated domains could be run instead, if ready. Most likely the 'next' thread on the same unit, and worry about repacking free slots the next time the schedule runs.

++ Added ++

It might be possible to have operations that don't cross security boundaries have different performance as operations within a program's space.

An 'enhanced' level of protection for threads running a VM like guest code segment (such as browsers) might also be offered that avoids higher speculation operations.

Any operation similar to a segmentation fault relative to that thread's allowed memory accesses could result in forfeit of it's timeslice. Which would only leak what it should already know anyway, what memory it's allowed to access. Not the content of other memory segments.

> However for threads from completely unrelated domains could be run instead

This introduces HT side channel vulnerabilities. You would have to static partition all caches and branch predictors.

Also this is more or less how GPUs work. It is great for high throughput code, terrible for latency sensitive code.

>if Intel or AMD announced a new gamer CPU tomorrow that was 3x faster in most games but utterly unsafe against all Meltdown/Spectre-class vulns, how fast do you think they'd sell out?

I do realize that gamers aren't the most logical bunch, but aren't most games GPU-bound nowadays?

Not a gamer but I would guess it depends on the graphics settings. At lower resolutions, and with less lighting features, etc. one can probably turn a GPU bound game into a CPU bound game.

I believe it does, that's why they test gaming CPUs at 1080p even when using the fastest GPU.

I wonder how that would play with anti-cheat systems, if the system has known vulns.

I imagine this is more of a functional issue. i.e., the loop buffer caused corruption of the instruction stream under some weird specific circumstances. Spectre and Meltdown are not functional issues but rather just side channel issues.

This should be fun, however, for someone with enough time to chase down and try and find the bug. Depending on the consequences of the bug and the conditions under which it hits, maybe you could even write an exploit (either going from JavaScript to the browser or from user mode to the kernel) with it :) Though, I strongly suspect that reverse engineering and weaponizing the bug without any insider knowledge will be exceedingly difficult. And, anyways, there's also a decent chance this issue just leads to a hang/livelock/MCE which would make it pointless to exploit.

A lot has been learned. Unfortunately, people still make mistakes and hardware will continue to have security vulnerabilities.

Complex systems are complex?

Sadly you're right. And obviously we're not about to give up on high IPC. I get it and I'm not judging -- it's just a bit saddening.

We learned that processor manufacturers love "bugs" that get solved by making them or your code slower giving you incentive to buy a newer one to restore the performance.

This seems unnecessarily cynical. Are you saying Intel/AMD are intentionally crippling CPUs?

I'm not saying Intel intentionally limited CPUs, just that they have intentionally limited a lot of things and lied about it in the past.

https://www.ftc.gov/news-events/news/press-releases/2010/08/...

Intentionally adding bugs? No. Intentionally making them slower with microcode and other software updates? Yes.

I was told the lesson is to avoid Intel and only buy AMD because they don't make mistakes.

No one said to buy AMD because they don't make mistakes. AMD just currently makes a better product overall.

I do not think you are accurately recounting what people said.

Was told by whom?

I'm still not convinced most of the computers in my home need to care about leaking data this way. I'm open to being persuaded, though.

I am not convinced either but I am willing to bet some software is adversarial and will try to exfiltrate data. F.ex. many people look suspiciously at Zoom and Chrome.

So as long as stuff is not perfectly isolated from each other then there's always a room for a bad actor to snoop on stuff.

For most of these vulnerabilities the risk is low, but keep in mind that your web browser runs random untrusted code from all over the Internet in a VM with a JIT compiler. This means you can't rule out the possibility that someone will figure out a way to exploit this over the web reliably, which would be catastrophic.

"Attacks only get better."

Right, but there's presumably little data to worry about exfiltrating from, say, my xbox. Even stuff like billing info doesn't need to be stored locally.

We should have learnt from the fdiv bug[0] that processor manufacturers need to be mandated to recall faulty hardware.

[0] https://en.wikipedia.org/wiki/Pentium_FDIV_bug

It depends on the severity of the problem, and the impact on the customers already using these systems. It may be more economical for the customer to apply a patch and lose a few percent of peak performance than to put thousands of boxes offline and schedule personnel to swap CPUs. This is to say nothing of the hassle of bringing your new laptop to a service center, and taking a replacement, or waiting if your exact configuration is unavailable at the moment.

This might come as a shock, but I can assure you that the designing high end microprocessors have probably forgotten more about these topics than most of the people here have ever known.

Huh?