First things first: Kernels panic and processes crash. If your medical equipment or telco/ISP system can't recover from that then you're in trouble anyway. Why they crash doesn't really matter in that context.

As far as voting machines go, kernel panic sounds waaay better than executing malicious code.

> Imagine a security f*ck up, like Heartbleed, but this time with an option to halt kernels / systems.

IIRC heartbleed didn't allow you to execute code (it allowed you to read more memory than you should have been able to). A better example is every flash player bug ever. Would you rather that thing crashes or executes malicious code? Keep in mind that the malicious code can also shut down your system.

Also keep in mind that we're talking about userspace programs right now. This thread is about kernel bugs. Userspace programs already have the option to ask the kernel to kill them if they misbehave. A lot of them do that (using features like seccomp filter) and many more should. (Chrome and Firefox both use seccomp filter I think.)

I see. It's ok because we'll just pass the buck and make it someone else's problem.

Let's say somebody gives you an USB stick and you plug it into your laptop. Which of the following scenarios would you like to see?

1. 0-day in the kernel's USB code. You're part of stuxnet now.

2. 0-day in the kernel's USB code. You're part of stuxnet now. You also get a message that tells you how and where to report the bug that was exploited.

3. 0-day in the kernel's USB code. Your computer crashes. You're not part of stuxnet. You also get a message that tells you how and where to report the bug that was exploited.

Linus is wrong (it happens). Exploit mitigation techniques aren't debugging tools. They're exploit mitigation techniques. The fact that they also produce useful debugging information is secondary.

This is exactly the kind of thinking Linus is talking about. In 3, I lost my work. Possibly very important work. To most people, being a part of stuxnet, while undesirable, is preferable to losing their work.

And you neglected a scenario 4: nobody is attempting to compromise my machine, but a buggy bit of USB code just crashed my system and took all my work with it.

You never learned at school to save what you are working on often? It's crazy, you are either too old to have required computer for school work or too young to have not lived through years of constant bluescreens.

Both 3 and 4 are mitigated by you saving your document often... it's not so bad, considering it can happen whatever you do.

Nowadays, Word is made to keep saving your change for that reason... They learned and designed it for the worst situation which is a whole system crash. If you can't handle that, well you aren't doing your work well.

Again, passing the buck. You know what I do instead of use your software that crashes all the goddamn time? I use someone else's software that doesn't.

Yes, of course we should save often, have decent backups, etc. But nobody is perfect and shit happens, and it'd be nice if the software you use didn't intentionally make it worse.

The problem is, what actually happened (in a previous commit) was:

The IPv6 stack does a perfectly sensible and legal thing. The hardener code misunderstands the legal code, and causes a reboot.

That it was Linus is worried about -- often it is hard to tell the difference between "naughty" code which can never be a security hole, and genuine security holes.

They should all be fixed ASAP, but making code that previously worked make a user's computer reboot, when it is perfectly fine, is not a way to make friends.

Bugs in the hardening code are obviously bad and annoying but that's besides the point. All bugs are bad and annoying, especially ones that cause a kernel panic. I don't think anybody is going to argue with that.

That's not what Linus said though. What he said is:

    > when adding hardening features, the first step should *ALWAYS* be
    > "just report it". Not killing things, not even stopping the access.
    > Report it. Nothing else.

and:

    > All I need is that the whole "let's kill processes" mentality goes
    > away, and that people acknowledge that the first step is always "just
    > report".

"Not killing things, not even stopping the access." Oh boy.

Step back a bit: when developing a new selinux policy, won't you develop first on permissive mode, and only after it's working without warnings, enable enforcing mode? It's the same thing here: the hardening should be developed first in a "permissive" mode which only warns, and then, after it's shown to be working without warnings, changed to be "enforcing" (in this case however, after some time the "permissive" mode can be removed, since new code should be written with that hardening in mind).

I didn't mean that to sound like I'm in favor of turning the thing on right away.

(Also, the quotes I chose don't really help me make my case but I don't want to edit now since you've already commented on it. His first mail is way worse: https://lkml.org/lkml/2017/11/17/767)

Basically what I'm disagreeing with is that exploit mitigation's primary purpose is finding and fixing bugs. That's just not true. Its primary purpose is to protect users from exploitable bugs that we haven't found yet (but someone else might have).

By first step, Linus just means "for a year or two". Yes it would be nice to put super high security on today, but instead we slowly turn up the setting, from opt in to opt out to forced on, to ensure we don't break anything.

4. 0-day in the kernel's USB code. Your computer crashes. You're not part of stuxnet. You also get a message that tells you how and where to report the bug that was exploited, but the part of your computer that was supposed to log the message died with the rest of the system, so you never see it and the bug never actually gets reported. Your computer continues to crash randomly for the next few days as an infected computer keeps trying to spread.

> What happens when a security bug stops the ventilating machine of a person lying in hospital bed, or halts the screen of a surgeon.

Linux is not a kernel for this kind of uses. Whoever does it, is doing a disservice to the people.

Operating systems like INTEGRITY RTOS or similar, are the only ones able to match the security quality requirements for such deployments.

https://www.ghs.com/products/rtos/integrity.html

"Linux is used in so many critical systems"

It is? I learned that in those areas you use different, much simpler and therefore more stable operating systems.

> Linux is used in so many critical systems.

It shouldn't be. It's ill suited for that.

Look at seL4, minix3, echronos instead.

Well apparently even minix3 is not free from critical vulterabilities ;)

https://security-center.intel.com/advisory.aspx?intelid=INTE...

Intel's shitty apps running on Minix != Minix itself

Minix 3?

Hooold it. Some of those things are not like the others.

--

I pity the engineers working on ventilation machines and the like. Medical devices are insanely hard to get right; that's neck and neck with aviation testing. I'm reminded of SQLite3's "aviation-grade" TH3 testsuite, which apparently has 100% code coverage. Let's be honest; Linux's monolithic design can't really attain that.

I would never use Linux for a medical device. I say this as someone who just happens to only be running Linux on every machine in the house right now (and I have for years, it's just how things have worked out, it's not at all novel or whatever, my point is that I'm totally comfortable with it). I'd use L4 or something instead. In a pinch I'd use a commercial kernel with tons of testing. Maybe I'd even use Minix; I'm quite sure a lot of people in industry are seriously looking at it now Intel have pretty much unofficially greenlit it as a good kernel (lmao).

--

Voting machines, on the other hand; I'd totally use Linux for that, because the security/usage model is worlds apart. Here, I WOULD ABSOLUTELY LIKE FOR THE TINIEST GLITCH TO CRASH THE MACHINE, because that glitch could be malware trying to get in.

The user experience of a voting machine is such that you walk up to it, identify yourself, and push a button. Worst case scenario in this situation is that you do some involved process to ID yourself and then the unit locks up, so you have to redo the ID effort on another unit. That is, for all use cases, not going to be a problem.

(I think that's the first time I've used all caps in years!)

--

Telecom systems... those are also a totally different world. See also: Erlang. In this situation you would likely want a vulnerability to literally sound a klaxon on a wall, but have the system still keep going.

I'm reminded here of an incident where a country's national 3G system was compromised (not the US, somewhere else) by hackers and the firmware of the backend systems was hot-patched (think replacing running binary code - the OS allowed it, it was REALLY hard to even notice this was happening) to exfiltrate SMS messages and cause calls to certain numbers to generate a shadow call (which ignored mic input) to an attacker-controlled number as well.

Telecoms is a classic case of massive scale; nowadays a single telecom switch might be routing thousands of calls through at a time. Yeah you don't want even a single machine to go down. But you DO want VERY thorough debugging, auditing and metrics.

(Which apparently don't exist.)

--

As for a Heartbleed-esque catastrophe, apparently one is going to be announced for Intel ME at the upcoming Blackhat(?) conference in December. I can't wait to hear about it myself.

Erlang's error-handling model is good (and interesting). Th e motto is: "Let it crash".

Each node does not handle errors at all, but PANICs on a fault. It is up to the supervisor (with global knowledge and state) to handle the fault appropriately.

It's good for uptime, but not good for correctness. The main problem is that it is hard to differentiate expected from unexpected crashes. Something like a missing pattern match can lead to a crash and it is very hard to know if the programmer "intended" for a crash to occur in that case or if the missing pattern is a bug.

You can have processes that reboot once every few minutes running for years because people didn't realize they were bugged.

>As for a Heartbleed-esque catastrophe, apparently one is going to be announced for Intel ME at the upcoming Blackhat(?) conference in December. I can't wait to hear about it myself.

Light on details, but the vulnerabilities are disclosed and fixed [0]. ME updates are already available from many OEMs.

[0] https://security-center.intel.com/advisory.aspx?intelid=INTE...

Right. But "don't apply the patch!" is sort of circling as well, because (presuming the Blackhat disclosure is workable, it sounds like it will be but fingers crossed) we might be able to play with our MEs.

Many medical devices run Linux. Most (AFAIK) patient monitors run Linux; GE and Philips (the biggest is business) both run on Linux. Those are the devices that keep you alive during surgery, make sure that those who are born too early (I don't know the English term here) are doing ok, monitor you state while you are in ambulance etc.

No...

Many medical devices run Linux as a User-Interface... (or Windows for that matter).

The actual safety-critical portion of these systems is rarely running Linux, but rather on a bare-metal micro.

That makes a lot of sense.

I'm reminded of a UAV doing the same thing. It ran L4 for low-level control, realtime scheduling, and security, and then virtualized Linux on top of that.

Sounds unbelievably clunky on the surface, then you realize it's a remarkably useful way to abstract everything cleanly.

Born prematurely..

For safety-critical systems, resetting on a fault is very much factored into the worst-case response time and expected behaviour.

PANIC on fault is exactly what you design into the systems.

What you find is that the truly safety-critical portion of the system is running on a microcontroller and the UI (which is not safety-related) can run on Windows or Linux.

Thank you for that!

Seems like so many in security fail to see the DoS implication.

There is no solution to bugs other than fixing them. And that's what Torvalds and others have been saying: for a security researcher, finding the bug is the end of the job. For developers, that's just the start.