Writing an OS in Rust: Catching CPU Exceptions

Re:Writing an OS in Rust: Catching CPU Exceptions by amluto on Saturday May 28, 2016 @06:57PM (#11793944)

IMO a bigger issue is that most of these kernel entries have unusual calling conventions. Exception handlers need to preserve all registers, for example. On x86_64, it's more complicated due to GSBASE handling. (Linux had it subtly wrong literally forever. OpenBSD still had it wrong last time I checked, but other hardening measures seem to make it impossible to exploit.

SYSCALL is even worse -- when the handler is invoked, there isn't even a stack.

[ Parent | on HN ]

Re:Writing an OS in Rust: Catching CPU Exceptions by bogomipz on Sunday May 29, 2016 @01:02AM (#11794883)

I'm not familiar with the term GSBASE, could you explain?

[ Parent | on HN ]

Re:Writing an OS in Rust: Catching CPU Exceptions by bogomipz on Sunday May 29, 2016 @10:33AM (#11796485)

I looked it up - segmentation in long mode. I'm curious how both Linux and BSD had/have gotten it "wrong" for so long? That just sounds unusual and likely not an oversight. Can you elaborate?

[ Parent | on HN ]

Re:Writing an OS in Rust: Catching CPU Exceptions by amluto on Tuesday May 31, 2016 @07:40PM (#11811139)

It's not segmentation. It's effectively just one extra register usable only for address offsets.

Unfortunately, on x86, privilege transitions and exception handling is an incoherent mess, and, worse, privilege changes (via IRET) can fail. Most kernels messed up what happens when IRET fails for obscure reasons, resulting in corrupting the value of GSBASE and therefore allowing a malicious program to change the targets of certain kernel memory accesses.

The result was "BadIRET", which I discovered, fixed on Linux, and reported via CERT (not a good experience) to other vendors. Someone else named it, and a few other people wrote exploits based on it.

[ Parent | on HN ]