> its purpose is to request a region of memory that is mapped only for the calling process and inaccessible to anybody else, including the kernel.

Easy fix: at the next context switch, the kernel overwrites the code of the process with its own code that copies the contents of the "secret" memory to somewhere the kernel can read it, and then executes its own code.

I guess I don't understand the threat model in which having data protected from the kernel is helpful.

If the idea is that the kernel is malicious, then it's not going to work in the first place (e.g. the kernel could just fail to honour the MAP_EXCLUSIVE flag in the first place). If the idea is that the kernel is initially trustworthy but gets compromised some time after the memory is made "exclusive", then it can execute its own arbitrary code in that process's context and get the contents that way.

I think the threat model is a buggy kernel - or a kernel insuffiently hardened against buggy processors with speculative execution side channels - with a malicious process.

E.g. if syscall S reads kernel memory without properly checking pointers/offsets (maybe GCC optimized them out), then process A can possibly use that to steal process B's secrets. Or maybe the kernel does check the pointers/offsets, but isn't hardened against yet another speculative execution side channel attack your processor is vulnerable to, so leaks some information via timing information.

MAP_EXCLUSIVE breaks that exploit, and to unbreak it you need both syscall S and another buggy syscall (or maybe another row hammer attack) - to write kernel memory - and to successfully leverage that into tricking the kernel into remapping process B's pages.

No panacea, but I could see it rasing the bar in exploit difficulty as part of a defense in depth strategy.

It does seem weird to protect user space memory from the kernel but not the memory of kernel subsystems from other kernel subsystems.

But at that point we're at a microkernel design that needs context switches between kernel processes... Good, I'd welcome that. This feature just implements a level of security that the whole kernel needs to catch up to before it makes sense.

I think you are getting at the same question Alan Cox had. As quoted in the article:

IMHO the question is what is the actual semantic here. What are you asking for ? Does it mean "at any cost", what does it guarantee (100% or statistically), what level of guarantee is acceptable, what level is -EOPNOTSUPP or similar ?

I think different semantics as Cox mentions would correspond to different threat models. With the right hardware support, it could mitigate against a kernel that falls under an attacker's complete control after the memory has been allocated. With less support it could mitigate against an attacker who can exploit read accesses at kernel privilege. But the details seem a bit open still...

It could also just access the physical memory via the direct map (the 1:1 mapping of physical to kernel virtual address space). In fact, this is kind of easier now, since you're guaranteed to find all the interesting stuff in physical memory since MAP_EXCLUSIVE memory cannot be paged out.

This is why the article says MAP_EXCLUSIVE would exclude the relevant pages from the direct map.

Ugh, I missed that. But there is still nothing preventing a malicious (nvidia) driver from re-adding the pages back. In fact, since they are missing from the direct map, it points anything malicious right at the interesting physical addresses..

I am thinking windows, so perhaps things are a bit different in linux, but most of your drivers run in kernel mode, that's a lot of code contributed by a lot of different companies that is fully trusted with all you can eat access.

The threat model is basically another process compromising the kernel. Think security systems in shared environments.

It's supposed to be hardware based. If process A requests memory with MAP_EXCLUSIVE, and it verifies that it got it that way, the hardware is supposed to make sure the memory is only available to that particular process. No amount of context switching is allowing the kernel to read it, nor could should it be able to modify the running code in a process if it's properly hardened (e.g. put the code to run in the protected area). It pose some interesting challenges, but it seems quite possible

This seems weird - the hardware has no notion of processes, that's a construct implemented by the kernel. It can enforce a separation of privileges between kernel and non-kernel, but that's about it.

Hardware gets a flag that says "am I in privileged mode now?" and it does not (and fundamentally can not) know to whether the currently running code belongs to process A or something else, or to whom a particular memory region belongs - as far as hardware is concerned, there's no "whom" at all, that's an abstract construct arbitrarily introduced and maintained by the OS.

While the current crop of mainstream CPUs does not support anything like it, the idea is old. Basically the CPU need to have hardware support for a PCB and be solely responsible for context switching, and the already complex MMU needs to be context aware.

I'm pretty sure we've seen specialized CPUs in the past that did exactly that.

That's how it works right now on x86.

There's no reason hardware can't be given a notion of processes, and used to set boundaries between each other and the kernel. It's not an untested idea.

It's about providing some defense against buggy and/or untrusted kernel driver modules maintained out-of-tree.

An exploit in your closed-source nvidia graphics driver should not get access to your private keys.

But it still will do if the nvidia graphics driver can override the code of your process, even if the nvidia graphics driver can't directly read the protected area of memory.

I guess it's a protection against kernel memory disclosure bugs.

AMD has this: https://github.com/AMDESE/AMDSEV

SEV is an extension to the AMD-V architecture which supports running encrypted virtual machine (VMs) under the control of KVM. Encrypted VMs have their pages (code and data) secured such that only the guest itself has access to the unencrypted version. Each encrypted VM is associated with a unique encryption key; if its data is accessed to a different entity using a different key the encrypted guests data will be incorrectly decrypted, leading to unintelligible data.

IBM z-mainframes have complete hw assisted total encryption in z14 and later. Keys are not visible to hypervisor, OS or application (code and data) and it's authenticated. Databases, datasets and network traffic can be transparently encrypted using hardware without altering applications.

AMDSEV/SME or Intel SGX/TME/MKTME or whatever other HW encrypted page support exist are buggy, which cannot be patched. E.g. AsyncShock

https://github.com/vschiavoni/sgx-papers

This is a SW-only solution, and therefore is probably also buggy, but can be patched and only accessed via ring < 0. Only via yet unknown kernel bugs (patchable) or the mythical ME or ASP/PSP backdoors.

https://semiaccurate.com/2017/06/22/amds-epyc-major-advance-...

And Intel apparently has MKTME, as mentioned in the link for possible hardware implementations: https://lwn.net/Articles/776688/

How about where you need some data that is available to all processes running under the same UID, but not to other processes? How do people handle that nowadays, especially on a generic server with no special cryptographic or security hardware?

To make it more interesting, assume that (1) the processes do not have a common ancestor running as that UID, and (2) if all processes with that UID exit, the secret should still be available if any new processes are started with that UID, and (3) the secret must be forgotten when the machine shuts down or loses power.

Assume that the secret should be safe from attackers who can run arbitrary code as other non-root users, who can force the machine to reboot, and who can steal the machine itself. (Stealing the machine causes it to lose power--no need to worry about a George Costanza/"Frogger" scenario).

For example, imagine a service implemented as a CGI that needs the keys for an encrypted database it uses, or needs the password for a service that it needs to access, using Apache using suEXEC to run this service's CGI under a dedicated UID.

It would seem that you need some kind of persistent storage. The first thing that comes to mind is a file. It can be owned by the UID of the CGI, and mode 0400. This doesn't meet the requirement of the secret being forgotten on shut down, though.

Would good old fashioned System V shared memory work for this? It has ownership and access control similar to files, so a shared memory segment owned by UID and mode 0600 should only be accessible to processes with the right UID (and root). It goes away when the machine shuts down. Linux's System V shared memory implementation adds a flag you can set via shmctl to keep it from swapping it.

It sounds like you want a file on a ram disk. Linux creates a ram disk by default in /dev/shm. You could create a file or directory there only accessible by a specific UID.

yes to shm. the use case you’re asking about is long solved.

It'd be interesting to have the kernel transparently encrypt the secret page whenever the process is put to sleep or waiting for IO, so that the contents are only readable when the process is active. Could be a nice, if not perfect, layer of protection against having the secrets read from other processes. IIRC there's a way to store crypto keys in CPU registers, so that they won't be available in a RAM dump.

You'd have to find a way to pin the keys in the registers, lest they be put in RAM during a context switch.

That's OK if your threat model trusts the kernel and only wants to protect against speculative execution attacks within the application itself. While the application is running, they keys will only ever be accessed from registers -- context switches are transparent to the application.

Yes, but there are debug registers that can be used for this.

https://news.ycombinator.com/item?id=11022275

Since MAP_EXCLUSIVE pages are implicitly MAP_LOCKED & MAP_PRIVATE, how much are they expected to be used in practice on a given system?

If it's an edge use case, couldn't they just use huge pages for such mappings to avoid the whole page-splitting performance problem in the kernel's linear mapping? Or only resort to splitting after some amount of huge pages have been wasted on exclusive mappings?

> Somehow, Bottomley suggested, the kernel should make the best choice it can for how to protect secret memory

Security can't be a best-effort platform-dependent QoI issue.

What could work would be several tiers with certain minimum guarantees that will either be fulfilled 100% or EOPNOTSUPP.

If a developer doesn't know what level of security they need or want, the feature is not for them.

Does it relate somehow to Intel SGX (Software Guard Extensions): https://en.wikipedia.org/wiki/Software_Guard_Extensions

What would happen to the data on suspend or hibernate?

Without consideration, presumably whatever mlock would do to it. So either it would prevent hibernation or those pages would be cleared?

Anybody remember Intel MPX?

I can't figure out why there doesn't seem to be a commercial market for something like CHERI ( https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ ) ... even if it was considerably slower than top end CPUs there are plenty of applications that don't care (and find chips like arm a8 or atom fast enough) but where high security is paramount.

Maybe Intel still remembers the pain of iAXP 432 but that doesn't explain the rest of the industry.

ARM is doing CHERI-enhanced cores, at least as a research project: https://www.eetasia.com/news/article/CHERI-based-Prototype-t...

There's also Dover Microsystems, which is trying to commercialize similar-in-spirit processor extensions originally developed at Draper Labs.

Sadly yes, while ARM and SPARC bounds checking get used to tame C and its derived languages, Intel decides to retire MPX.

Apparently MPX had a number of issues in general that made it worse than tools like Address Sanitizer: https://arxiv.org/pdf/1702.00719.pdf

I know, but then the path forward would be to fix it, instead of dropping it on the floor without any plan going forward.

After all, SPARC ADI is used in production, and ARM as well (partially with iOS, enforced in future Android devices).

What feature does iOS have that that is similar to this?

Pointer authentication and hardened runtime.

Those don’t really catch memory safety issues in the general case; they’re more focused on exploit mitigation.

As far as I can tell, Intel MPX is like a hardware address sanitizer. This is a little different.