News for hackers. Stuff that matters.
Re:Self-Crashing Cars by Animats on Monday January 29, 2018 @01:21PM (#16259444)
Real problem, clueless author.

The big problem is allowing firmware updates.

Many years ago, I knew some of the people involved with the Ford Electronic Engine Control IV, the "EEC IV", used in 1980s Fords. This used a custom CPU, an Intel 8061, which was an Intel 8051 with some extra timer features. The program was etched onto the CPU silicon. There is no way to change that program without replacing the whole unit. There's an external ROM with some tables, different for each engine model, but it's a true ROM, not something that can be rewritten. If you want to replace it, it takes a wrench.

The level of paranoia and testing that went into that program was very high. Any bug meant bringing back hundreds of thousands of Fords in a recall. That didn't happen, and there are still many EEC IV vehicles running today, 30 years later. That's what it was like in the days of hardcore embedded programming.

I'd argue that safety critical vehicle software should not be downloadable. If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system.

Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)

Re:Self-Crashing Cars by mindslight on Monday January 29, 2018 @01:44PM (#16259624)
The deeper problem is Internet access. Firmware in ROM can still have remotely-exploitable security bugs. That Ford team only had to test for states that the software(/hardware) would end up in on its own, rather than defending against malicious external stimulus pushing it into a previously-unexplored state.

Given that it seems like a foregone conclusion that these things are going to end up with some WAN interface [0], pure ROM is out of the question. The best we could probably do would be to assert that the flash could only be updated through a local debug port, with protections against eg full RCE allowing remote re-flashing.

Given physical access, it's trivially easy to sabotage a car - the new problem here is having it done remotely. More important would be some kind of auditing so that a local debug device could be assured of reading out the exact code image, knowing how many times it had been flashed and ideally checksums, and being sure that a reflash reset the system completely.

(btw, loved "shit early, shit often")

[0] I mean, I'd personally rather they not. But it seems like a foregone conclusion of trends/marketing/sloppy design/implementation shortcuts/can't stop the signal.

Re:Self-Crashing Cars by jacquesm on Monday January 29, 2018 @01:59PM (#16259757)
True story: I looked into a company in the vehicle automation field that had OTA upgrade on and didn't bother to check whether or not the vehicle was in motion or not during the upgrade. You really can't make this stuff up.
Re:Self-Crashing Cars by 3pt14159 on Monday January 29, 2018 @09:16PM (#16262394)
It’s so much worse than just that jacquesm.
Re:Self-Crashing Cars by jacquesm on Tuesday January 30, 2018 @06:36AM (#16264485)
I did not mean that to illustrate rock-bottom, merely as a datapoint. Yes, things are much worse than that. But this one is pretty easy to understand without further context.
Re:Self-Crashing Cars by ohazi on Monday January 29, 2018 @01:28PM (#16259480)
My car is >10 years old, and thankfully still running like a champ. It'd be nice to get a new one someday, if only for the updated safety features like curtain airbags, but I honestly don't feel very comfortable with anything much more recent, for the exact reasons you've described.
Re:Self-Crashing Cars by reaperducer on Monday January 29, 2018 @02:29PM (#16259978)
My inner curmudgeon is on board with the idea of only allowing important firmware updates to happen at the dealer. But the reality is that with so many cars on the road, OTA update is a better way to make cars safer.

In my case, for example, I have a 2014 and a 2015 car which are both more than a year overdue for a software updates. One is related to the braking system; the other to a problem where if you stomp on the gas in an emergency (like to avoid a crash), the car stalls.

The problem is that the dealers in my area don't have the capacity or the will to do software updates. One dealership only has maintenance hours between 9am and 4pm Monday through Thursday, and the ONE guy who is allowed to do software updates is only in on Saturdays from 9-noon occasionally.

The other dealership doesn't take appointments at all. It's first-come, first-served, 8am-5pm Monday through Saturday. I stopped going there because even at 7am, there's a huge line of people waiting for service. The final straw was when I sat there from 7am until 3pm for a very simple non-engine part swap that was covered under warranty.

The next nearest dealer is 138 miles away.

tl;dr version: OTA updates are good because there are places where you can't get a software update done at the dealer in a reasonable time.

Re:Self-Crashing Cars by Bilters on Monday January 29, 2018 @03:14PM (#16260274)
If we’re talking about the self driving cars here like mentioned in the article, the cars could go to a dealer / workshop on their own. You’ll only have to tell the car “I don’t need you for the next x hours”. So you go ahead and have the firmware updated.

Unfortunately there’s no quick fix about the current cars having the “guided” self driving principle.

Re:Self-Crashing Cars by Piskvorrr on Tuesday January 30, 2018 @05:22AM (#16264200)
Unfortunately, the current climate is quite opposite, what with computers randomly* deciding "you don't need me for the next couple of hours, I'll try the install-revert-install-revert-install-revert with this week's update, NOW."

(randomly, as in "where do you want to go today? meh, nevermind.")

Re:Self-Crashing Cars by ScottBurson on Monday January 29, 2018 @05:06PM (#16261064)
Please tell us what make of car that is, so we can avoid it.
Re:Self-Crashing Cars by reaperducer on Tuesday January 30, 2018 @12:21PM (#16267029)
Fiat.
Re:Self-Crashing Cars by ams6110 on Monday January 29, 2018 @01:50PM (#16259663)
> I'd argue that safety critical vehicle software should not be downloadable.

I'm not sure I'd go that far. OTA "automatic update" probably not a good idea. Downloadable/flashable at the dealer OK, no need to go as far as swapping out ROM modules.

Following avionics software engineering standards should be absolutely required. Entertainment/nav systems that require network access should be 100% air-gapped from the vehicle control systems.

Re:Self-Crashing Cars by sigstoat on Monday January 29, 2018 @01:31PM (#16259514)
> Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving.

ehhh, cars shouldn't be communicating with cars they can't see. but i'd like to them to communicate over free space optical, aimed at the level of other vehicles. brake lights could be modulated to contain current braking force, and reason for braking. turn signals could contain the destination lane. head/tail lights the current speed, and target speed.

Re:Self-Crashing Cars by Animats on Monday January 29, 2018 @01:39PM (#16259572)
Communicating a few bits over radars is a possibility, like the way TCAS works for aircraft. There's a simple fixed-format message, agreed upon internationally, which is used to negotiate "I go up, you go down" decisions to avoid collisions. One advantage of data over radar is that you have range and bearing info on who's talking to you. At least you know where the info is coming from.
Re:Self-Crashing Cars by wmf on Monday January 29, 2018 @01:34PM (#16259536)
None of that information can be trusted, so maybe it's better to not even bother.
Re:Self-Crashing Cars by 05 on Monday January 29, 2018 @02:51PM (#16260139)
Neither can be turn signals, yet we still use them
Re:Self-Crashing Cars by cortesoft on Monday January 29, 2018 @01:45PM (#16259630)
Information that can't be trusted can still be useful.
Re:Self-Crashing Cars by mindslight on Monday January 29, 2018 @02:13PM (#16259880)
Neither can brake lights.
Re:Self-Crashing Cars by marcosdumay on Monday January 29, 2018 @04:36PM (#16260828)
Self-driving cars can work without trusting brake lights, and I think they should. It's humans that can1t use anything better.
Re:Self-Crashing Cars by Piskvorrr on Tuesday January 30, 2018 @05:24AM (#16264208)
If only it wasn't for those pesky humans, all of this wouldn't be an issue. [blocky 8-bit lightbulb appears over robot's head]

Seriously, "let's have everyone jump on the bandwagon all at once, and we shall be redeemed by $TECHNOLOGY_DU_JOUR" is a very brave proposal.

Re:Self-Crashing Cars by mtgx on Monday January 29, 2018 @01:56PM (#16259731)
Tesla can already change its cars' performance with OTA updates. You know what that means, right? It means that all software can be changed with a single OTA update. And much of this system's security depends on how secure Tesla's servers are.

How cars were designed 30 years ago is completely irrelevant, because that's not how all or most new self-driving cars will be designed. They will be designed so that the software can be changed on the go.

Re:Self-Crashing Cars by PinguTS on Monday January 29, 2018 @01:30PM (#16259503)
> Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)

You seem to be out of touch what happens today. Car2Car or Car2Infrastructure is about as aircraft to aircraft communication is used in TCAS. It is about what car (object) is at which position, so that collision may be detected as early as possible to prevent them.

Also next generation cars will have their maintenance modes locked down as far as possible. OBDII will be closed down as far as possible. All those dongles, which are hip right now, will be rendered useless. But of course the tuning scene has still a lot of incentives like in the past to thwart all this and to reverse engineer. It will be a cat-and-rat game like it always was.

Re:Self-Crashing Cars by Symmetry on Monday January 29, 2018 @04:09PM (#16260629)
I found this critique of those fairly convincing.

http://www.templetons.com/brad/robocars/v2vdata.html

Pithily, "It can never be that useful because if it's being really useful, something is very wrong."

Re:Self-Crashing Cars by 3pt14159 on Monday January 29, 2018 @01:47PM (#16259638)
Because the cars are self-driving they require updates to improve the autonomous software that controls them, including improvements related to security.

As for surveillance related to V2V communications, I fail to see how it meaningfully adds any surveillance over just monitoring the other electronic emissions from the car like those that emanate from the media centre or even the phones of those within the car.

Re:Self-Crashing Cars by wyager on Monday January 29, 2018 @01:51PM (#16259670)
> Because the cars are self-driving they require updates to improve the autonomous software that controls them

This doesn’t follow.

> including improvements related to security.

This should not, under any circumstances, be a reasonable justification. Cars should not have any external attack surface, period. The ECU and any other critical electronics should not be networked at all. Unfortunately, that ship has long since sailed.

It should be possible for cars to be updated, but it should require physical access, and any mandatory update (e.g. due to a safety failure) should result in the manufacturer incurring heavy costs, e.g. you have to give every owner $1000. We have to internalize the externalities of shitty software, especially in stupendously dangerous machines like cars.

Re:Self-Crashing Cars by 3pt14159 on Monday January 29, 2018 @02:00PM (#16259763)
I don't understand why it doesn't follow. If there is an adversarial attack that causes the autonomous software that controls the self-driving cars to crash then they need to be updated.

> This should not, under any circumstances, be a reasonable justification.

That presupposes that humans cannot make mistakes, which they do. Whether it is hackable software defined radio, or just plain oversights.

The levels of abstraction that programmers deal with do not allow for the necessary security you propose.

> Cars should not have any external attack surface, period.

What you advocate for is a car with no ports, no radios, and a perfectly secure component manufacturing chain. It isn't doable. There is a reason the ship sailed.

Re:Self-Crashing Cars by singingboyo on Monday January 29, 2018 @02:08PM (#16259847)
There's a very simple update mechanism for any car (or anything that doesn't get put in space or dropped to the bottom of the ocean, really). Recalls.

They're expensive, annoying, etc. But they're safe (or safer, at least), and if you screw up badly enough that a safety-critical system needs an update, they're necessary.

I don't think anyone is arguing that the infotainment system should be disconnected from the world, but the core driving systems (steering, brakes, transmission, etc) should be air-gapped and inaccessible without physical access to the interior of the car. Also, that access could (and maybe should) be disabled when the car's engine is running, preventing an attacker from plugging in a wireless device and then subverting the system with the car in motion.

Re:Self-Crashing Cars by icc97 on Monday January 29, 2018 @05:51PM (#16261339)
I agree that that's a better solution. But in what utopia is that going to happen. It would have to be government mandate to stop all over the air updates because no single company would take the extra cost when the competition isn't. This would only be done by a government after an attack takes place.
Re:Self-Crashing Cars by singingboyo on Tuesday January 30, 2018 @04:33PM (#16269253)
Well, in a true utopia it'd be unnecessary, because no one would hack the cars. Which would be nice, but... unlikely.

In any case, I agree that it's unlikely unless there's a severe disaster related to OTA updates, but it's still the best solution from a security and safety perspective.

Re:Self-Crashing Cars by icebraining on Monday January 29, 2018 @02:13PM (#16259877)
no ports, no radios

I don't think so, just that the radio and other non-critical components mustn't be connected to the critical components. Essentially the radio becomes like any other portable device, that just happens to be physically installed in the dashboard.

Re:Self-Crashing Cars by wyager on Monday January 29, 2018 @06:48PM (#16261674)
> If there is an adversarial attack that causes the autonomous software that controls the self-driving cars to crash

If it is discovered that any model of car is vulnerable to this, it needs to be recalled and scrapped.

> The levels of abstraction that programmers deal with do not allow for the necessary security you propose

This is true of the kind of shit pumped out by low-quality commodity hardware and consumer-focused software companies. Anything controlling millions of multi-ton missiles should ideally be formally verified top to bottom. Exceeding NASA vehicle code standards are the sort of thing that would be appropriate here. However, just the plain old standards we apply to the embedded code in ECUs plus full hardware isolation (i.e. no “start from your phone” bullshit) would be sufficient.

> What you advocate for is a car with no ports, no radios,

No radio attached in any way to the critical hardware is sufficient. The goal here is to defend against bulk remote attacks (the kind that are actually dangerous in a noticeable economic sense) and bugs, not evil maids.

Re:Self-Crashing Cars by bnj on Monday January 29, 2018 @02:08PM (#16259836)
If anyone cares to, I’m curious about this and it would be great to benefit from your insight. I haven’t had anything to do with autonomous systems, but from my layman perspective it seems as though there would be legitimate benefits to inter-vehicle communication assuming a sufficiently sophisticated network / platform—

For example, cars being able to broadcast their immediate path and do some dynamic route mapping based on which lanes and areas will be in use by the time it arrives; being able to agree with nearby vehicles about routes that are partially colocated so they could join together bumper to bumper for that leg and realize fuel efficiencies (I have no idea if that would actually be useful but I’m brainstorming possibilities); being able to negotiate a car parking space in advance of arrival...

Is it that there actually aren’t many applications where inter-car communication is useful, or is it more that the benefits of communications like these are outweighed by risks to security, or am I missing important pieces of the puzzle?

Re:Self-Crashing Cars by cosin on Monday January 29, 2018 @08:11PM (#16262117)
The second one.
Re:Self-Crashing Cars by amelius on Monday January 29, 2018 @04:35PM (#16260822)
> I'd argue that safety critical vehicle software should not be downloadable. If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system.

Better yet, the software should be uploaded to an intermediate agency, that is responsible for the testing of cars (could be part of the US Department of Transportation for example). Only this agency can (after sufficient testing) upload the software to the actual cars.

Re:Self-Crashing Cars by blattimwind on Monday January 29, 2018 @02:01PM (#16259778)
With most car brands you're lucky if the car repair shop even bothers to tell you when they installed an ECU update. There is, obviously, no change log available anywhere. Sometimes people whipser behind backs, though.