Who are these sinister organizations? Chaos? E.V.I.L.? Or the FSB or what?
A terrorist attack? Wouldn't you be more worried people going crazy and driving down the wrong side of the highway?
By your thinking maybe we shouldn't even have cities because they are dense target rich environments. Even under an attack by Loki or whatever sinister organization from a comic book, you're still talking about something far safer than what we have currently.
I think it's hilarious that HN is the first place to see people saying the Boston Bombing was no big deal and more babies are killed each year by peanut buffer or something but yet here we have people more concerned about Magneto controlling cars and causing terrorist actions than reducing the very real 95% of traffic deaths. That's pretty hard to take seriously!
We already live in an age where pacemakers with wireless components can be hacked, and potentially kill the people who wear them.
And remember, it isn't just potentially the government who might do this. What about hackers working on behalf of some malicious and sinister agency? What if instead of kidnapping you, they mess with the controls and cause you to drive into incoming traffic? Or worse, an entire highway's worth of cars in a terrorist attack?
I'd be first in line to buy an autonomous automobile, but these are still real concerns. The more control you cede to programming, the more rigorous the code has to be.