Interesting that AWS has been mitigating for side channel attacks since before they became a big news item. Curious about Azure and GCP's stance on this

Or, maybe, they just thought the lack of deterministic performance created billing/accounting/customer service problems. (One hyperthread can just about completely starve in many circumstances).

"Are We Susceptible to Rowhammer? An End-to-End Methodology for Cloud Providers"

https://arxiv.org/pdf/2003.04498.pdf

Although their answer in this paper was diplomatic, my interpretation is that they confirm it as a problem. Their conclusion was it would not be as bad it was considered at the time. To be revisited on the context of this more recent work.

Edit: Adding main reference

"BLACKSMITH: Scalable Rowhammering in the Frequency Domain" https://comsec.ethz.ch/wp-content/files/blacksmith_sp22.pdf

You pick a VPS because you want to save costs and share cores.

So it’s not so much AWS choosing as you the customer choosing.

I ran the numbers back in 2015 and persuaded the previous employer to go for dedicated tenancy with all performance-critical and privacy-sensitive workloads. I was effectively hedging against unknown but practically guaranteed cross-VM attacks to pacify a paranoid regulator.

Then Rowhammer happened. Less than a day later, our contact with the regulator comes to me asking how it affects us. Being able to answer - with absolute confidence - that it did not, was one of the proudest moments of my career. And the turnaround from "regulator comes asking awkward questions" to "regulator is happy and sees no reason to ask again" of less than 48 hours must be some kind of record too.

[0]: https://aws.amazon.com/ec2/physicalcores/

The looming thread of side-channel attacks on SMT systems has been known since well, before we had SMT systems (because it also can apply to Co-Processor, and non SMT multi core systems).

The difference between back then and today is "we believe it's possible but haven't found a way yet" and "there are multiple known ways", as well as it being wide spread known instead of just in some communities.

The reason we still shipped the problematic CPU's is because improvement in perf. and as such competitiveness and revenue on the short term where more important.

There also was a shift in what people expect from security and which attack vectors are relevant. For example user applications a user installed where generally trusted as much as the user. While today we increasingly move to not trusting any applications even if they are installed by a trusted user and produced by a trusted third party. Similar running arbitrary untrusted native code from multiple untrusted users and "upholding" side-channel protection wasn't often an important requirement in the past.

I think the big difference is that we have attacks with high bandwidths of exfiltrated information-- instead of very slow, difficult to control leaks.

Sure, but this is worse than that. This is "your online poker game client can gets access to your web browser's bank account session info."

We need process isolation within a single machine, or else we are kinda screwed as a field.

But this is an electrical problem. Interference is a huge issue with any engineering that involves physical things and these kinds of attacks are just interference problems. This issue is no different from a microwave knocking out your Wi-Fi. These attacks have become possible because the acceptable interference threshold that chip makers have been using has turned out to be too low.

How do you fix interference problems? First, you choose a new threshold of acceptable interference and then you engineer better isolation, you lower density, and/or you switch technology.

We could make shared computing complete safe tomorrow if we wanted to so I think calling this the end to shared computing is quite alarmist. The issue is that we collectively want to both have the cake and eat it too: we currently have a certain cost-to-compute ratio that we have become accustomed to and we don't want to compromise that. We're basically buying time until we can invent a new technology that can achieve the same density without the same level of interference.

So maybe "screwed as a field" is not an exaggeration if the field is butt computing.

Think about someone trying to do basic ETL. Like having a tabular file and summing it or something. Don’t use Excel, we say, stand up a $4 million Spark and AWS architecture with seven hundred pitfalls that can let bored Russians take over your whole network as if they were going to the dry cleaners because remember, you just might be Google someday. That’s where we are. It’s been a complete industry failure for a decade and it’s only getting worse. Accelerating, even: now you need some operationally-terrifying Kubernetes to even be at the table, and then as an industry we (rightly) say running this stuff ourselves is too hard, so pay Amazon to do it rather than even ponder if we have settled on the right approach.

Tada: Humanity just lost computing to three companies. We very likely aren’t getting it back.

There are probably 5,000 people doing this work who can adequately secure such a system and make it mostly impermeable. Where middle computing is royally screwed is that nearly all of them work in San Francisco or its clones abroad. So then you get “best practice” blog posts and industry think pieces and the lowest bidder ties them together into something resembling a competent computing system. That’s been the state of the art since 2004 everywhere except Santa Clara county.

With the exception of some areas in the IC and DoD, I just described the entirety of US government IT. That ETL example? It’s actually real and underpins a small part of Medicare across several government contracts. Because the tools the valley exports are all they’ve got, and we sure love building systems with massive footguns, and then shaming organizations publicly for missing item #543 on the tribal “secure your computing system” checklist and shooting themselves.

The entire industry must change, top to bottom, but just like climate change, again, that’s a nonstarter. Posix and the Web are not the path forward and I hope I live to see the industry figure that out. I’m increasingly skeptical. The good news is my hometown might flood into the sea first, sparing me from considering in my last moments that every argument I’ve ever made in this profession has fallen on deaf ears and that everyone has to derive our industry’s peril from first principles for themselves.

I think it will come down to what you are willing to call 'individual' processors. But actually having physically distinct memory seems like a lot of overhead for attacks that won't matter for 90% of users. Also I would think that the on-board ECC of DDR5 would protect it against these types of attacks.

The trend is yet to come. My statement is that, if AMD/Intel doesn't adapt, Amazon has the hardware investment to leave them behind, just like Apple did.

But to be clear on two points: They will probably adapt. And Amazon/etc will probably never leave them behind fully. DCs, especially public cloud, are not all-or-nothing like Apple's Mac Lineup is.

Then the question follows, why would they want something Intel/AMD isn't offering right now? The trend is System-on-Chip. Beyond Security (this isn't the last electrical interference/speculative execution-like attack we'll see). SoCs are easier to service (easier != cheaper. holistic replacement versus per-component debugging. servers are cattle, not pets). Denser. More vertically integrated. Capable of far higher IO performance. Lots of benefits.

Mega-servers with 256 cores and 4 terabytes of memory still have a huge place in all DCs; but not when multiple untrusted workloads are running simultaneously. They're not for EC2/Fargate/Lambda/etc; they're for S3. Highly managed, trusted workloads.

And indeed to my understanding spectre, meltdown, rowhammer and similar attacks are not an issue there.

https://en.wikipedia.org/wiki/Z/Architecture

I wonder when more features from the mainframe cross pollinate Intel amd arm CPU architectures.

A lot of the talk of mainframe levels of security is specious at best.

https://www.suse.com/de-de/support/kb/doc/?id=000019105

Meltdown didn't affect z nor amd.

https://www.zdnet.com/article/meltdown-spectre-ibm-preps-fir...

https://wiki.debian.org/DebianSecurity/SpectreMeltdown#Syste...

https://tenfourfox.blogspot.com/2018/01/actual-field-testing...

If I understand correctly, Homomorphic Encryption aims to solve for these kinds of attacks (although presumably the computations are more expensive and programs must be restructured to use HE primitives?). https://en.wikipedia.org/wiki/Homomorphic_encryption

EDIT: Why the downvotes? Am I mistaken?

Not to mention, to avoid side-channel attacks, an HE scheme still needs to always run the longest possible sequence of operations regardless of input (otherwise, information about the input data is leaked through timing). So an HE version of a quick-sort scheme would always run in O(n^2), otherwise it would leak details about the contents of the list. In some cases it would even have to run in the same amount of time regardless of the size of the list, to avoid leaking information about that.

This is still phenomenally slow, of course.

It works a little bit like this: you write your algorithm using only HE computing primitives, then you encrypt it and the input data and send those to the HE runtime, who you don't trust, and who DOESN'T have any way to decrypt your message. The HE runtime executes your encrypted program and sends back the still encrypted output. Once you receive your output, you decrypt it using the key you have kept secret.

I'm not sure what you mean. This is how HE works, at a high level. The whole point is that you're doing public operations on secret data on a machine that can never know what the data was, nor what response it gave you. Furthermore, the point of HE is also that you know that your data remained secret regardless of how the HE runtime was implemented, as you just send it encrypted information.

Making it more efficient by giving it access to the decrypted data would defeat the whole point. Making it more efficient by optimizing based on the actual data would allow your data to leak through timing side-channels, again defeating the whole point.

Basically, HE refers to computing schemes where DECRYPT(ALGORITHM(ENCRYPT(X)) == ALGORITHM(X). You send [ALGORITHM, ENCRYPT(X)] to the runtime, and it gives you Y. DECRYPT(Y) is then the result of your computation.

As far as we know, this is indeed hopelessly slow, and there is no guarantee that we will ever find fast secure HE primitives.

And yeah, this is just as inefficient as you'd think.

Firstly the computations on encrypted data just take a LOT longer, especially for Fully Homomorphic encryption. With a single 64 bit addition taking microseconds.

Secondly, the FHE code cannot branch based on data. If it could, it would know something about the data, and it wouldn't be proper encryption. This means an if statement becomes "calculate both branches, and throw away the result you don't need". Similarly, a for loop becomes "give me an upper bound of how long this will run", then "loop for exactly the upper bound number of times, if the loop is 'done' early, just throw away the result of the remaining operations".

FHE is really cool, and has its uses in situations where you want to cooperate without needing to trust, but it is stupidly inefficient. (Things get really cool if two parties want to cooperate and the computing party can e.g. branch based on their local unencrypted data)

But it's likely things like different core counts, frequency performance curves, perhaps more or different memory/IO controller counts, perhaps instruction extensions of some particular interest (I'd wager BF16 was available in data center SKUs long before consumer SKUs), etc.

It's nothing radical, but if you're going to buy an awful lot of them and want something slightly custom, Intel will definitely do that for you.

Cloudflare mentioned they use a custom Intel SKU on their blog, and they're not as big as AWS.

Your take sounds accurate for non-FAANGs (and a couple of those). For Google, not so much. There is a lot of custom acceleration and such that isn’t available to any other customer, and a nearby commenter is right, every detail about them is locked behind a very strong NDA. I’ve heard thirdhand AWS does indeed have custom instructions in the virtualization extensions stuff, but don’t know if that’s true.

Right you are, Ken.

Depends on how many you order and how reasonable the customization is. Like, what if I wanted a 10,000 38 core Xeons with a 2.5GHz Base clock and 3.5GHz Turbo clock? Such a part doesn't exist, yet sits smack dab between two existing Models, the Xeon Platnium 8368 and 8368Q. Assuming yields are already good enough, that might be the sort of thing a CPU maker will do. (Might need more than 10k units though, IDK.)

Intel does not want to be competing with an Amazon Basics processor.

"The custom Epyc 7003 part that Facebook has commissioned from AMD has 36 cores out of its potential 64 cores activated, and with a bunch of optimizations for Facebook applications at the web, database, and inference tiers of its application stack."

While this seems to be mainly about Power/Heat/Perf. optimizing for typical "cloud" workloads it might also have design decisions related to this problem. We will see, but they look interesting anyway.

I'm curious about the eventual end-game of security in this space. Take the 64 individual processors in your example, give each one their own independent memory bus to their own ram chip, isolate them from each other as much as possible. What else can be done, if a malicious process on processor Z has to go all the way to disk to try to get back at data working on processor J, is that as maximally secure as it can be without being in a completely separate chassis with only network access to the other device?

Essentially, a board with an array of "mini computers", 1-4 cores bussed to their own physically and logically isolated memory pool. Storage is oftentimes already networked in public cloud, and seemingly less susceptible to these kinds of attacks, so probably little change is necessary there.

Its also likely we'll see vendors produce more "card computers", like the raspberry pi, but oriented at a server environment. At massive scale like what AWS runs at, these are less compelling, but they do have advantages; the biggest being, maintenance is a cinch, and the blast radius on one failing is isolated to just the workload on that card. I imagine these will be more interesting for current VPS providers like DigitalOcean, who also want to offer dedicated hosts with smaller core/memory counts. Even RPis today are super compelling; a $50 multicore computer that's easy to manufacture and replace, with density high enough to fit 100+ inside a 4U? Just need a bit more core perf and ARM adoption (though its easy to see Intel also hoping on this train with their long-term investment in pro/consumer NUCs).

Writing is on the wall. The tipping point will be an industry thought leader saying, in effect, "you should be using dedicated hosts, because at this point physics is our enemy and we're not winning, by the way here's a new dedicated host instance type that's perfectly suited for horizontally scaled workloads, its 20% more expensive than virtual hosts of the same size, byeeee".

I have server application capable of utilizing many threads and thousands requests/s. You think I will deploy it on tiny 2 core CPU? No thank you, it currently runs on powerful dedicated server from Hetzner where I control everything.

>"AMD and Intel need to adjust, or face extinction ....

If they don't evolve, AWS will for them..."

Sounds like pontification / FUD.

Sounds like you didn't fully grasp what the grandparent was writing about.

>> Eventually, it will be obvious that running shared workloads on a single piece of physical hardware has fundamentally unremediatable security implications

> I have server application capable of utilizing many threads and thousands requests/s. You think I will deploy it on tiny 2 core CPU? No thank you, it currently runs on powerful dedicated server from Hetzner where I control everything.

Your use case isn't covered here, you're neither a cloud nor VPS customer.

There will always be individuals and companies looking for dedicated hosting solutions, or just hosting it themselves.

However, for the rest of the world that wants to run their stuff in the cloud that's where Amazon's and other companies investments in ARM come into play. That's where we'll see disruption, the extinction the grandparent was talking about.

Unfortunately it will likely have negative performance implications for multi-tenant work loads.

That being said, and even if I consider that a quite different subject, I agree that the current efficiency story of Intel is not very good, but I hope they will improve in a not so far future. The dev lifecycle of CPUs is quite long and it seems to be an obvious target. I suspect they will be forced to improve their efficiency, because that's actually were the performance potential is today (the current dissipation level of their last desktop CPUs is not reasonable, and prevents scalability). And trying to lower the core count also can yield to high consumption, e.g. if you want your performance back by increasing the frequency. Wide and "slow" is needed, and it is harder to increase the internal number of execution units per core and have them actually used, than to increase the core count -- plus ironically one way to do that is through HT, which goes against your wish to share less hardware. (Now if you compare their P-core and E-core in Alder Lake the story is more complicated, but their marketing figures seem very strange, so I won't conclude anything for now. The current instances of P-core we have is for that weird desktop market with unreasonable high TDP anyway.)

Now if you really want miniaturized individual computers that would not be shared at all, I'm not sure the market will actually go into that direction because big systems will continue to be needed (and clusters are a niche mostly for HPC), and I'm not sure a "slightly more secure on smaller systems" market would be interesting enough. Esp. in an era of chip shortage. And also because it would still be bigger than a shared equivalent system make with the same techs... But if that's really a niche that has to be addressed, I suspect it would mostly be a matter for Intel to create new small and slower SKUs ("slower" compared to their desktops insanities) -- they even kind of have that already, but yes the physical miniaturization aspect is not handled yet -- that does not really depends that much on the cores, though. And even in those computers, I'm not sure there would be much demand for very low core counts. The threading of pretty much all workloads tends to increase, nowadays.

One last point, after the e.g. Pentium 4 fiasco nobody really left the IPC race. AMD had some difficulties when trying "weird" ideas (part of which were because of their marketing communication), and then again a completely new design from scratch to market takes time. In general there was a pause of performance growth around 2016 for a few years, and that was mostly Intel having process problems and the rest of the industry catching up (and then overtaking them).

For me cloud computing is just where the best pay is. I do not at all see it as the future of computing.

One reason is ML will help us realize we write code we don’t need; so much of it is syntax sugar for business specific needs; infra, security… it’ll be realized cloud software is solving unemployment not technical problems of value. That many issues with software back in the day were lower quality networks, and consumer hardware. I mean any phone can abstract metadata from any one users amount of behavior, we do it in a DC because that’s where the jobs are. Chip manufacturing will include ML normalized logic for specific application.

LAN IOT will improve and we’ll realize the Metaverse can be implemented with a local client and AI generated art, on a mobile GPUs power in a few years. Middle men like Zuckerberg face the most uncertain future. He failed to diversify as well as Bezos, Newell, and others.

IMO, Valve is a serious threat with Steamdeck; an open IOT brain in a kid friendly form factor could be the new cigarette. Even Apple may have to take them seriously. My kids iPads need replacing soon; a flat glass slab with no interactive controls, requiring another $800+ machine to develop on, bloated development tools, fees, and a bunch of cloud logins, are not going to motivate kids to feel creative.