Timing leaks are an absolutely fundamental flaw of computers. It may be the case that this specific instance of speculation-based attacks that leak information via timing channels can be fixed by stopping speculation or by complete rollback of speculated state including caches. "It's the chip's fault" is true for Spectre and Meltdown, yes, and it will take some time for Intel and other manufacturers to design new chips that stop leaking information due to speculation.
However, timing attacks are probably everywhere. They are covert channels that subvert language safety. The question isn't whether information leaks, but what and how fast. How are we going to deal with them comprehensively at the language level? I don't know.
Indeed, having no speculative execution, or just having caches separated by trust domains, would make Spectre impossible. This would make the whole promise of 'securely virtualized hardware' largely disappear, though. For guaranteed isolation, you'd have to have a physical core (or set of cores) per trust domain.
With trust domains sliced pretty thinly, e.g. a domain per JS-executing web page, it's hard. AFAICT, reseting the caches when switching processes (that is, trust domains) sort of mitigates this attack at a relative low cost, a performance hit instead of having to replace the CPU.
Wrong.
Even if you separate different privilege levels onto different physical computers, and they only exchange information by message passing, then they are likely to leak information merely from the message timing.
The message timing may indicate what has been cached or pre-computed on the privileged machine. So it's a matter of sending messages from the malicious machine such that they manipulate what is cached/pre-computed on the privileged machine based on the contents of privileged information.
Manipulating what is cached/pre-computed is generally not too hard for an attacker; the real trick is doing so based on the contents of privileged information. But it's conceivable that you could extract some information from a database by determining which pages of a btree got cached when you try (and fail) to look at something you're not supposed to. It might be hard to extract the 1000th byte of a big text field that way, but you might be able to find out whether "joe" or "sally" is the person you're looking for.
Sounds a bit difficult in practice, but computing (perhaps more than any other field) tends to quickly eliminate the difference between theory and practice when the motivation is strong enough.
The more you trust the code you run alongside other code, the less precautions like that you have to take. Delimiting security domains right is hard, though.
On a related topic, I know someone who worked on CPU/FPGA hybrid architectures. The idea was to move processes between CPUs and FPGAs. Awesome stuff, realtime processes offloaded to the FPGA get to run completely off the CPU, with ridiculously low latencies. The issue is that you really have to design your code for it.
How about 16 big core and 4096 small cores that don't have speculative execution?
https://content.riscv.org/wp-content/uploads/2017/12/Tue1136...
Perhaps one day we'll talk about how many cores are allocated to a program rather than the other way around. OTOH even their chip will have shared caches. Spectre is achieved via speculation but communicates via side channel of the cache, so it's not clear this is a whole lot better.
What nobody wants to say is that the real problem is running untrusted code. They don't want to say it because Apps and JS are so many peoples bread an butter these days.
Course, in the context of the larger thread, I can't help but wonder how much more attack surface this would tend to create.
We could consider something like Tilera chips working together with privileged "general compute" chips, the same way as current machines work with separate GPUs as privilege "vector compute" chips.
Ok, fine, in some abstract sense. But they are fundamental to realizations of computing that involve shared resources, as the rest of your comment is getting at.
We want to share resources for efficiency, which is going to be the constant pressure that keeps leaks coming.
Reality is always about compromises and tradeoffs. Now we see how the balance point is being pushed back closer to the pole of "security". A new equilibrium will eventually appear.
(You can see the very same process in e.g. encryption algorithms: tradeoffs between performance and security, and phasing out of solutions that yielded either to finding of logical flaws or increases in available computing power.)
As far as sharing cache goes, there can be 4096 PCIDs/ASIDs AFAICT, maybe there should be more. Tying speculative execution to PCIDs/ASIDs might not be too bad of an idea.. if it isn't done already. Better CPU security models that reason about cache and speculative execution should be enough to solve this problem.
Disable TSC access, lower clock_gettime precision, make clock_gettime return fuzzy (randomized) time. Randomize CPU frequency to make usage of spinning threads for time measurement hard. Insert randomized delays into untrusted binaries, kinda like DTrace inserts probes (i.e. imagine a JavaScript JIT that inserts no-ops all over the compiled code, and randomly changes these to random sleeps). etc.
A practical vision of computing without these things is challenging, though!
At least on the server side, that's realistic. Big companies like Google already run their own trusted code on their own hardware. For everyone else, it's not impossible to move from VPS hosting to "bare metal cloud" like Scaleway offers.
(Vision for the Future™: instead of their tied-to-Linux "someone else's Raspberry Pi" experience, it would be a bunch of tiny single core processors, each connected to their own tiny DRAM chip. You upload unikernel images or CloudABI binaries (that would be then wrapped into unikernels) into the Cloud™, where the load balancer boots up your images on-demand on as many of these tiny computers as needed. RAM is zeroed before boot.)
The other big environment with tons of untrusted code is of course the web browser. This is significantly more ridiculous, but: why not embed a ton of tiny simple in-order ARM cores without any shared cache into desktop processors, and run JS/WASM on them? (One core used per origin per site) :D
I definitely chuckled. :) On a more serious note- I'd bet at least a couple bucks it would be possible (albeit super weird and difficult) to channel information through hardware-acceleration mechanisms. Lots of rendering happens on the GPU these days.
rr does just this! I think it uses the "instructions retired" performance counter as its "fake time"; that turns out to be deterministic enough for its purposes. Whether it's deterministic enough in a security context, I don't know for sure.
But this approach, though it can run threaded software, will not let untrusted software (which should really mean all software!) use more than one real core or hardware thread.
Only if you define 'deterministically' to mean 'determined only by the program's own legitimately accessible context'. These timing side-channel attacks work precisely because the processor performs in a deterministic and detectable manner, but does so based on information which the attacking program should not be able to access.
The program is equivalent to a pure function from (state, events) to (state, actions) and the computing device that evaluates this function can't leak any information to the function without evaluating it incorrectly. It can still leak information in how long it takes to evaluate the function, but there is no direct way to get this information back into the function.
You could ask the user what time it is, or how long something took, but that starts to look suspicious fast!
You just need to refine statistics. The underlying program can‘t be solved in a physical universe.
If the randomizations have such an effect that data sniffing is still possible but its speed is 1 bit per week, trying to sniff something interesting becomes impractical. (Or not; depends on your threat model.)
Given that "silicon is cheap" why don't we work around these problems by ensuring that processes within different security domains run on separate cores with their own cache and branch prediction cache? At context-switch, flush all that.
This is similar to eval(). The operating system always needs a way to load new code, but you don't need it in the language, or least not in a sandboxed language.
It's also more consistent with the abstract programming model that we typically use. A new version of a function is normally considered compatible with the previous one based on the values it returns, not the details of its timing (within reason).
We often depend on this correctness vs. performance separation to be able to make performance improvements easily. It's very convenient to be able to swap in faster code without worrying that you'll break any callers. It's also convenient not to have to write constant-time functions (to avoid information leaks) most of the time.
Industry reality is that we are still fighting stack overflow bugs 40 years after safe languages were invented.
Timing attacks are going to be around a long time.
As devices become more powerful and more connected, thus increasing the attack surface and the potential losses form them being cracked, the cost-based analysis will start to prioritize security more.
The tools for writing dramatically less brittle code are there, but there's the cost of switching. It needs to become lower than the potential losses. And with the current state of the industry, the losses and lawsuits are surely coming. Remember what it took to make an average e-commerce website become reasonably secure.
Getting people to use heavy static analysis is an uphill battle. Getting people to use prepared statements to prevent SQL injection is an uphill battle! But nobody should take then lesson that we are woefully unprepared to deal with timing leaks via program analysis and that we've entered some new unfamiliar territory.
For example, here's Z in C++14, used to compute 5 factorial: https://godbolt.org/g/se5B8u
I think proving that no recursion exists should be relatively simple, just keep following the call graph and keep a set of the nodes (functions) called; you either reach the terminal node(s), or reach an already-visited node again, which means a recursion exists.
[0] https://en.wikipedia.org/wiki/Total_functional_programming
But sometimes it is hard, and I'm afraid it's equivalent to the halting problem in general case.
Without recursion, though, finding the maximal depth of the call graph is definitely solvable, and likely not even very hard.
Side note: an interesting case is the Stable Marriage Problem. It's easy to write the O(n) algorithm and prove its bound, but it's hard to translate into a bounded recursive algorithm that does the same thing.
(At least as far as I understand -- I searched it on Stack Overflow and that's what it said, and I've tried the problem itself. A Coq fanatic insisted that "if you can prove the run time is bounded it's easy to write the bounded recursive equivalent" but somehow that never translated into an actual insight on how to do it.)
There are still cases where you can prove that a recursion is bounded (I suppose something like dependent types could help here). There are definitely cases where you can prove that a loop is bounded, because the semantics of the language requires them to: a loop with a counter (not otherwise mutated), iteration over a known finite structure, etc.
Well, both C++ and Haskell allow you to use compiler features that are undecidable in general case, but usually work well in practical cases. Also, if e.g. your recursion / loop analysis is taking too long, you can just stop the analyzer. If the code can't be reasonably proved right, it should be considered wrong (with a chance of a false negative).
> That is, the basic premise of hardware process isolation is that a process cannot have any knowledge of what another process is doing, or whether it even exists,
Side channels such as the cache and timing break that promise.
This has been known forever. Nobody can contemplate a decent CPU without cache!
The thing that spectre and meltdown have demonstrated - obvious in hindsight! - is that CPUs that speculate can use this side channel to exfiltrate secrets in shockingly powerful ways that are really hard to defend against.
We can do things to stop speculated loads leaking via cache hoist/mru/evict/etc and make chips immune to these powerful attacks.
Eliminating the general cache side channel is a damn sight harder and still an open problem.
The cache attacks against crypto systems eg scrypt are fascinating just like spectre.
Low-latency memory access, low-latency conditionals, secure isolation ... choose two.
I like to contemplate a huge SRAM on die. Get rid of caches and speculation entirely. Of course that doesn't work for really big systems, but we might one day have a full 32bit address space available like this. Now hopefully you can contemplate it too!
I get your point and also your analogy of the "bricks::hardware" and "house::software".
Yes, Spectre has nothing to do with C/C++/Javascript/Rust safety. Yes, we can look at the world as it currently is with existing languages and confidently say that.
However, I think you're inadvertently creating a thought-stopping comment instead of charitably reading his essay as exploring a possible future of language security and whether that langsec evolution is practical or provable. That author starts that line of thought with, "Where do we go from here? I see but two options. [...]"
My attempts at addressing the author's possible future would be:
1) microcode instructions may be considered a "competitive advantage" and therefore cpu manufacturers like Intel and AMD wouldn't publicize them such that GCC/Clang could incorporate that intelligence into a compilation algorithm. There are already variations of this profit-vs-openness tension happening such as NVIDIA not releasing open source Linux drivers.
2) publishing microcode architecture and putting the optimization responsibility into into GCC/LLVM sounds very similar to the failed EPIC instruction set[1] for Itanium. There seems to be an underlying human law of "specialization" and having Intel hardware engineers figure out all the speculative out-of-order-execution was better for us than having Richard Stallman and his gcc colleagues work on it. Unfortunately, the hardware side of things created a 20 year old bug. Is there new evidence about the interaction of hardware optimization & compiler theory that won't repeat the Itanium EPIC failure?
3) having a completely specified "hardware model vm" transparent to a compiler and language spec doesn't necessarily remove side-channel attacks. It may just change who creates the vulnerabilities... e.g. the Clang LLVM team (instead of Intel) inadvertently creates a timing attack inside the target executable.
[1] https://en.wikipedia.org/wiki/Explicitly_parallel_instructio...
I'm not so sure at the root cause level, part of the problem is that programming languages treat memory as a contiguous blob and speculative execution is a work around for that. All languages are designed as if the memory bus speed and cpu are still identical. Maybe languages need better ways to tell CPU's what data they are going to need ahead of time so that the CPU can have it ready.
Could we build a cache aware programming language? Imagine something like an iterator that would process one item and concurrently pull in data for the next iteration. If creating correct hardware is too hard then maybe we can be more explicit at the software level.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc....
Formulated thusly, the author's argument seems eminently valid to me, and exposes the old bug-bear: the root of all this evil is nondeterministic execution. Elsewhere in this thread I bemoan the defunct IA-64 (Itanium) architecture because it delegated explicating inherent parallelism to the compiler and allowed the core to be shorn of rescheduling circuits, and (it now emerges) enhances security.
But that “hardware” is mostly a program running on a machine whose ISA you don’t know. Intel is shipping a microcode patch.
In an earlier life I wrote microcode, as well as programming machines whose ISA could be changed by rewiring the backplane (PDP-10). I used to think I “knew the machine” and, in today’s parlance, “the stack” as well. Not anymore — nobody can grasp the entire stack.
Unless your program is a VM for a programming language (e.g. JavaScript) that runs code from different origins in the same protection domain. Even perfect hardware process isolation won't help you solve that problem.
The price is steep, though — at least with the current crop of hardware.
That said, even a single VM per page doesn't help, because you have scripts from advertising networks running alongside other scripts. You'd need at least a VM-per-origin-per-page.
I suppose that once a page includes third-party scripts, it already gives them the key to its kingdom, like sniffing anything from DOM or altering it in arbitrary ways. Thus per-page isolation seems sufficient to me; maybe even per-page-set isolation when I open many pages from the same site (this needs a way of clearly defining what "the same site" is).
doesn't same origin policy prevent this?
Think about loading jQuery from a CDN, for example, which people do. If third-party scripts couldn't access the DOM it wouldn't be very useful.
I have had the experience of seeing the 'sausage making' of CPUs on the other side when I worked at Intel and the verification of the 80286 was a big thing and the evolution of the 80386 was just starting. And the sad truth is that hardware engineers, exactly like software engineers, are often compelled to accept a 'good enough' solution when the business decides they have reached that point of what the business is willing to invest against the expected monetary returns.
In software this will express itself as an algorithm that is correct 99.99% of the time. Or one which is computationally expensive and so replaced with an algorithm that takes less computation but gives good enough results. And in the normal course of things, those decisions are often "ok" because failures are rare and when they fail there isn't any "gain" by them failing.
Security of course turns that calculus on its head, where, if you can break the chain somewhere, you win. So a place where the chain is only 99.9% as strong as it is elsewhere will get special attention. And when the chain breaks there is gain that exceeds the effort involved.
For much of the behavior that is responsible for both Spectre and Meltdown, the CPU can be constructed to not bypass the other secure checks. If the designer chose to, they could have the CPU machine check or fault if it read protected memory in a speculative path. Even if that path wasn't actually taken.
I don't know of course, but I would expect the typical conversation at a CPU design house would be, this;
A) We should always fault if we access protected memory
B) But what if its just an uninitialized pointer on a branch that won't be taken?
A) It is still a fault.
B) Ok but really "only" if we take the branch so lets wait until the code is actually executed to fault. Tree in the woods with no one around to hear it and all that ...
Good engineers, designing cool machines, making completely reasonable decisions about what is 'good enough', against a spec that says "CPU will fault if it ever accesses protected memory." because it will fault if the code ever gets there. As opposed to designing the fetcher to always check memory protections and always fault. To any reasonable person, both would seem to be sufficient, but only one is correct. And since it takes more silicon and more complexity to always check, we get the one that is good enough. It is my hope that at this very moment there are CPU designers all around the world changing the HDL in their design to be correct, and verifying that they run just as fast as they did before, just that now your CPU can fault on an address that wasn't actually executed. I am looking for any example where speculative execution or fetching of protected memory is ever not a 'bug' waiting to happen in a piece of code.
As a result I expect CPU designs to be updated, and new micro-architecture where this behavior always faults but is otherwise just as performant. No doubt it will take some additional transistors.
Um. Allocate a page of memory and then write to it (e.g. memset(mmap(NULL, PAGE_SIZE, PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0), -1, PAGE_SIZE). Each time the CPU hits the loop condition in memset it will speculate the loop to continue and write to the next byte of memory (or more if unrolled). When it hits the end of the page the speculative execution will attempt to write to an unmapped page (or worse), but that's fine because the loop condition is falsw and so the write isn't retired.
If you can't speculate that a loop that has run 4096 times will continue another iteration, what can you speculate?
Yes, but safety is safety. The point of the article is that the assumption on the part of language nerds for the past few decades -- that correct choice of processes and implementation patterns as expressed by whatever favorite runtime the author wants can protect us from security bugs -- turns out to be FALSE.
Rust and Haskell programs are subject to Spectre attacks just as badly as K&R C ones from 1983. All that pontification didn't actually work!
The point is that it puts a limit on pedantry about "correctness". That's all based on an idea about a "computer" being an abstracted device, when it turns out the real-world machines are in fact leaky abstractions. The lesson to my ears isn't that you shouldn't use Rust or whatever, but that some humility about the boundaries of "correctness" would probably be a good idea for many of the pontificators.
This is a bit silly. Security is a chain that is only as strong as its weakest link. For decades we believed that C was the weakest link, and now, with Spectre, we have discovered (drumroll, please!): nah, C is still the weakest link. For all the amazingly scary things that Spectre brings to the table, it's still less scary than Heartbleed and Cloudbleed were.
Don't get me wrong, hardware still sucks (and with Intel and AMD's growing body of management engine incompetence they may yet overtake C as the weakest link in the chain) but we will learn from this and get better. In the meantime, we need to be pushing forward on all fronts: hardware security, language security, kernel security, application security, network security... nobody gets to stand still.
https://crypto.stanford.edu/~dabo/papers/ssl-timing.pdf
And even if C is the weakest link, all that means is that you're totally screwed. No amount of Rust will save you when the Linux kernel, your relational database, and every router between you and the attacker are written using C or assembly.
At the end of the day, if you want strong security assurances you need to prove the correctness of all the semantics (of which memory bounds constraining is but one small aspect) of every piece of software in your stack, or at the very least all the most critical pieces. And the best tooling for that is for Ada and C.
I'm probably biased, but in my experience the people actually pushing for correct-by-construction tools tend to be acutely aware of the limitations of their approach, especially when it comes to having to trust hardware to do what it says. For example, all of the people I've seen advocating for seL4 tend to be very transparent that their proofs assume hardware correctness and that they don't think it's realistic to exclude much of the hardware from the trusted computing base. Further, it seems to me that the Rust developers have been extremely pragmatic about what is possible to mitigate and what they have to trust the hardware for.
I'm sure there are some examples of what you're describing, but without any sort of references, the generalizations you're making have a whiff of bad faith.
Um, Rust and Haskell aren't about security against malicious attackers exploiting hardware/system flaws. They are about correctness against programmers writing (some classes of) bugs by mistake. The two aren't directly related. Program correctness is a worthy goal even if it doesn't protect you against hardware problems.
Same goes for the proponents of formal stuff like Coq. Even worse, because those languages are so insufferable that they induce design errors, the number of which will grow each time a poor soul will have to maintain the... thing. Then, all right, there is no mistake, the result maps perfectly to all your design errors.
> when in fact most of the serious information leakages do not originate from such problems
We've already established that the parent post made a mistake in thinking FP is about safety against attackers / information leakage. Why are you still talking about this?
> because those languages are so insufferable that they induce design errors
[citation needed]
What's your actual experience with these languages?
That is, the basic premise of hardware process isolation is that a process cannot have any knowledge of what another process is doing, or whether it even exists, unless it is explicitly permitted (e.g. the OS offers an API to look at processes).
This basic premise is now broken: enough information leaks via timing as to allow to read arbitrary memory regions. It's like you built a house laying bricks in a formally correct way, but the walls proved to be permeable because the bricks themselves have a flaw.
Since this flaw is not fundamental, that is, not even every currently existing processor exhibits it, it appears to be reasonable to replace the flawed design of the bricks, and not to re-engineer house construction.
There is a nice quote in the article, about doing basic science to find out how software (and hardware) actually works, since specs are badly incomplete. It can be matched with another, earlier famous quote: "Beware of this code; I only proved it correct, I did not test it." Black-box testing of information systems is not going anywhere, formal methods or not; at the very least, we need evidence that formal methods were not visibly mis-applied, and also evidence that other basic premises, like those of hardware, still hold. Because sometimes they don't.