Or what if an actor uses something like this:
https://techcrunch.com/2016/06/29/now-you-see-me-now-you-don...
that confounds the car's sensors. Self-driving cars have a whole spectrum of attack vectors that human beings don't have.
Yes, hostile input will have to be taken into account. Fortunately, this is easily detectable (just look at the logs). And if people die as the result, it will count as murder. Finally, one does not simply jam sensors from across the planet. You need a physical presence on site, and that's riskier than a remote hack.
A lot of the current driving issues do suck, but they still will suck with self-driving cars, as any computerized system has its own vulnerabilities. It's just they might be systemic and affect a lot more people in ways they can't compensate for.
Of course even machine checked proofs will have trouble finding vulnerabilities that creep in during the specifications stage or exist in the realm of hardware.
How do you isolate your sensors from external input and expect to do anything?
I did say "very expensive". But yes, if we're serious about correctness, safety, and security, I don't see any other choice. We should scrap the crap and spend the $billions necessary to rebuild it right.
Ideally. We don't really need to go that far. Isolation is possible and not that hard, see Qmail: http://hillside.net/plop/2004/papers/mhafiz1/PLoP2004_mhafiz...
> Of course even machine checked proofs will have trouble finding vulnerabilities that creep in during the specifications stage or exist in the realm of hardware.
I'm no hardware specialist, but you can still prove the correctness of the design of such and such hardware. The actual chips can still fail the specifications, but that's easier to test once you know the design itself is correct.
Also, specifications themselves can be checked. Only the high-level properties must ultimately be decided and reviewed by hand. Not that they won't be complicated or numerous, but that's still much smaller and easier to deal with than the entire implementation.
> How do you isolate your sensors from external input and expect to do anything?
I wasn't talking about the normal sensor input, which of course can be tricked the same way human sensory input can (for instance with a big flash or something). I was thinking about sensor command, such as which way they should be oriented or something. Though I expect most sensors will have no such input, and will only output to the system.
I used to think that was unlikely. Then I had a few discussions with people about the state of C, and things that could be done to make it default to a slightly more deterministic case by changing how undefined behavior is dealt with in regards to optimization. Now I think it's impossible, because nobody is willing to give up even theoretical unknown performance increases for more security. At least for C. You would have to use something that's much more strict about behavior, like Ada with SPARK, or possibly Rust.
I'm willing to bet most the code already written for these projects is C or C++. Good luck getting that changed if so.
Correct software is expensive. But if we make sure incorrect software is even more expensive, we'll get correct software.
Once the industry is forced to get serious about correctness, they will move away from the C/C++ minefield real quick βor at least come up with safe ways of using C and C++.
Correct software can be compromised via the update process, or social engineering. It's necessary, but not sufficient to ensure integrity.
A cheaper, correct option would be a mechanical lever/breaker labelled "manual override" for licensed drivers or "emergency stop" like they have on every industrial robot or heavy machinery since the 60's. Sometimes the answer to a software problem isn't more/better software.
Such malware are only possible because the targeted system has vulnerabilities to begin with. One just has to ensure the absence of such vulnerabilities, possibly using machine checked proofs.
One obvious approach is to properly isolate the driving software and sensors from external input.