Once you get a _good_ solution to embedding untrusted code, you might find it turns out to be pretty useful.
No, process isolation does not work for us, because it does not scale well enough.
The point of Cloudflare Workers is to distribute your code across Cloudflare's entire edge network -- 154 locations and growing -- so that you can respond to requests at the closest location to your end user. We want to put every customer's code in every location, rather than forcing you to choose a handful. Meanwhile, obviously, not every one of our locations is a mega-datacenter.
This means we need a way to support extremely large numbers of tenants per machine, with relatively low traffic per tenant (because each tenant's traffic is spread out over the world). We need this to be efficient.
This means, we need low memory overhead (to support many tenants per machine), very fast cold-start time (since cold starts are a lot more common in a decentralized scenario), and low overhead to switch between workers (because we're mixing traffic from all our customers everywhere).
Using embedded V8 isolates within a single process gives us 10x-1000x better performance on all these metrics than if we gave a whole process to each customer.
Relatedly, here's a talk I gave at Heavybit about the neat things you can do when you have fine-grained server compute: https://www.youtube.com/watch?v=YZSvJNBZsxg
https://v8.dev/docs/untrusted-code-mitigations#sandbox-untru...
We've spent a lot of time thinking about and building mitigations for speculative side channel attacks. For example, early on in the project -- before anyone even knew about Spectre -- we made the decision that `Date.now()` would not advance during code execution, only when waiting for I/O. So, a tight loop that calls `Date.now()` repeatedly will keep getting the same value returned. We did this to mitigate timing side channels -- again, even though we didn't know about Spectre yet at the time.
Chrome has indeed stated that they believe process isolation is the only mitigation that will work for them. However, this statement is rather specific to the browser environment. The DOM API is gigantic, and it contains many different sources of non-determinism, including several explicit timers as well as concurrent operations (e.g. layout, rendering, etc.). Side channel attacks are necessarily dependent on non-determinism; a fully-deterministic environment essentially by definition has no covert side channels. But, there's no way Chrome can get there.
The Cloudflare Workers environment is very different. The only kinds of I/O available to a worker are HTTP in, HTTP out, `Date.now()`, and `crypto.getRandomValues()`. Everything else is perfectly deterministic.
So, for us, the problem is much narrower. We need to make sure those four inputs cannot effectively be leveraged into a side channel attack. This is still by no means trivial, but unlike in the browser, it's feasible. `getRandomValues()` is not useful to an attacker because it is completely non-deterministic. `Date.now()` we've already locked down as mentioned above. HTTP in/out can potentially be leveraged to provide external timers -- but the network is extremely noisy. A practical attack would require a lot of time in order to average out the noise -- enough time that we can do a bunch of higher-level things to detect and disrupt possible attacks. It helps that Workers are stateless, so we can reset a worker at any time and move it around, which makes attacks harder.
NetSpectre demonstrated that even physical network separation does not necessarily protect you against Spectre attacks. There's simply no such thing as a system that's perfectly secure against Spectre, process isolation or not. All we can do -- aside from going full BSG and giving up on networks altogether -- is make attacks harder to the point of infeasibility. Luckily, we have lots of tools in our toolbox for making Spectre attacks infeasible in the case of Cloudflare Workers.
2. "BSG"?
The Battlestar Galactica was the only ship in the fleet that hadn't networked its computers, because the captain was paranoid. When the Cylons (AI) attacked, they instantly hacked all the other ships, but the Galactica stayed under human control and got away.
It's fiction, but I'm honestly really impressed with this bit of writing. It's both an entirely plausible and almost realistic strategy, and it gives the writers an excuse for the crew members to interact rather than let the computer do everything. Whereas on Star Trek one wonders why they bother with a bridge crew -- the captain might as well be punching everything into a computer directly rather than inefficiently giving orders to humans.
I'm comforted by the fact that if there's a breakout in V8, Chrome is a much juicier target than we are -- and has an incredibly strong security team that will jump on the issue. I think we would be overall less secure using a lesser-known JS implementation.
1. No process isolation 2. Data from lots of users on any given node
Am I missing something?
* While process isolation makes Chrome somewhat less interesting than it used to be, keep in mind that process isolation is being rolled out as a defense to arbitrary-read attacks (like spectre), not arbitrary-code-execution attacks. Executed code can potentially talk to the rest of Chrome and perform actions that a web site is not intended to be allowed to. (At least in the past, this included the ability to read all the user's cookies, though it's possible they've improved on that, I don't know.)
* Cloudflare has potentially multiple tenants in a node, but the attacker has no control over who they are. There's no particularly good way to target a specific end user nor a specific web site.
* V8 vulnerabilities are likely to be memory safety issues, which are difficult to leverage into an attack without an understanding of the process's memory space, which is hard to get without having access to the source code or a compiled binary. The attacker would be flying blind and would almost certainly just segfault or be caught by our layer 2 sandbox (which blocks almost all syscalls), which incidentally would immediately alert us to their activity. (I don't particularly like this as a defense, since I like open source -- but frankly, it is a pretty big barrier.)
* The attacker would be burning their zero-day by uploading it to Cloudflare. We block eval() and the like, such that the only way to run code on our edge is by uploading it through our deployment APIs, which keep a copy.
* Last I checked, Chrome still does not use site isolation on mobile, because the overhead is too high.
If anything the industry is increasingly adopting bytecode as distribution format, kind of returning to the early 60/70's designs with microcoded CPUs just with another approach to the final execution.
The real benefit is having a portable bytecode that 1) can accommodate any language out there without constraining their object and memory model, and 2) is guaranteed to be compiled to native code by an optimizing compiler when deployed. Add some basic APIs, and we could have a situation that has heretofore has been enjoyed pretty much only by x86 - the ability to compile code, and run it 30 years later as is.
But C, C++, Rust, Go, D, etc... all support compilation to those platforms. This is a really common thing to do.
Otherwise you are severely crippling yourself for no real gain. You're going to lag the hardware features by years if not decades, you're not going to get arch-specific optimizations as readily, and you're going to have huge startup costs. You're also stuck with sandbox restrictions that you've self-imposed, making it harder to do things that would have otherwise been trivial or needs to be re-invented.
You could maybe squint and claim you need a portable IR, but given there's only really 2 ISAs in widespread use (x86 & ARM) does that _really_ matter? Similarly you could claim cross-OS support, but again as there's only _really_ 2 OS's for each target (Windows/Linux for server, Android/iOS for mobile) does that _really_ matter? Critically does it matter to such a degree that you'd rather severely cripple your own app to get it instead of just compiling it 2, 3, or 4 times?