News for hackers. Stuff that matters.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by txcwpalpha on Thursday June 07, 2018 @02:03PM (#17258461)
>This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software.

Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that has a problem with this.

>So what is the right way to handle these updates?

The way that other vehicle manufacturers (car, airplane, etc) have been doing it for decades is a pretty good way.

>You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer.

There is no evidence that said OTA update made Tesla cars any safer. There is evidence that similar OTA updates have made Tesla cars more unsafe.

The brake OTA that you mentioned has actually potentially done more harm than good. Tesla owners have been reporting that the same update made unexpected changes to the way their cars handle/accelerate in addition to the change in braking distance. These were forced, unpredictable changes that were introduced without warning. When you're driving a 2 ton vehicle at 70mph, being able to know exactly how your car will react in all situations, including how fast it accelerates, how well it handles, how fast it brakes, and how the autopilot will act is crucial to maintaining safety. Tesla messing with those parameters without warning is a detriment to safety, not an advantage.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by omgwtfbyobbq on Thursday June 07, 2018 @04:01PM (#17259665)
>Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that has a problem with this.

The TACC offered by most (if not all) manufacturers can't differentiate between the surroundings and stopped vehicles. I wouldn't be surprised if their Lane Keeping Assist (LKA) systems have similar problems.

https://support.volvocars.com/en-CA/cars/Pages/owners-manual...

>WARNING When Pilot Assist follows another vehicle at speeds overapprox. 30 km/h (20 mph) and changes target vehicle – from a moving vehicle to a stationary one – Pilot Assist will ignore the stationary vehicle and instead accelerate to the stored speed.

>The driver must then intervene and apply the brakes.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by darkerside on Thursday June 07, 2018 @08:24PM (#17261563)
This comparison just sold me on how morally wrong it is what Tesla is doing. Intentionally misleading and marketing to customers a feature called Autopilot that is only a marginal improvement on what other cars already offer. What if Volvo started calling their (clearly not independent) feature Autopilot and saying it was the future of hands-free driving? Seems inexcusable.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by slg on Thursday June 07, 2018 @10:59PM (#17262246)
Which is also exactly what GM is doing with Super Cruise. Here is just one of their commercials.

https://www.youtube.com/watch?v=u__51kTl4j8

Despite this warning in the manual[1]:

>Super Cruise is not a crash avoidance system and will not steer or brake to avoid a crash. Super Cruise does not steer to prevent a crash with stopped or slow-moving vehicles. You must supervise the driving task and may need to steer and brake to prevent a crash, especially in stop-and-go traffic or when a vehicle suddenly enters your lane. Always pay attention when using Super Cruise. Failure to do so could result in a crash involving serious injury or death.

[1] - https://www.cadillac.com/content/dam/cadillac/na/us/english/...

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by TeMPOraL on Friday June 08, 2018 @01:29AM (#17262834)
Riffing off the parallel thread about Google AI and how "corporations are controlled by humans" and can have moral values - no, corporations are controlled primarily by the market forces. When Tesla started branding line assist as autopilot, it put market pressure on others to follow suit. Hence, I'm absolutely not surprised about this ad and the associated warning in the manual.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by darkerside on Friday June 08, 2018 @06:14AM (#17264041)
TBF, corporations are controlled by humans and overwhelmingly influenced by market forces, which are also controlled by (other) humans.

That's a nitpick. Your broader point about Tesla pressuring the market down an unfortunate path is spot on.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by omgwtfbyobbq on Friday June 08, 2018 @05:27PM (#17269851)
That branding started a long time before Tesla was ever around.

http://www.oldcarbrochures.com/static/NA/Chrysler_and_Imperi...

Ideally, yeah, every manufacturer would have to take all the puffery out of their marketing, or better yet, talk about all the negatives of their product/service first, but I doubt I'll ever see that.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by darkerside on Friday June 08, 2018 @08:42PM (#17270858)
https://arstechnica.com/cars/2018/06/why-emergency-braking-s...

This article portrayed Super Cruise as something qualitatively different, based on the maps of existing roadways. I'm not sure if they've also considered integrating the multiple systems involved in driver assistance. I'm curious if Tesla has either for that matter.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by thinkmassive on Friday June 08, 2018 @10:31AM (#17265752)
I’m opposed to over-regulation of any sort, however it seems obvious that vehicle manufacturers need to do a better job informing consumers of the driver assistance capabilities of modern vehicles. Something similar to the health warnings on cigarette packs.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by codedokode on Thursday June 07, 2018 @05:45PM (#17260574)
> The TACC offered by most (if not all) manufacturers can't differentiate between the surroundings and stopped vehicles.

Software should not be driving a car into any of them. I think that LIDAR would see the obstacle, but as I understand, the crashed Tesla car didn't have it.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by omgwtfbyobbq on Friday June 08, 2018 @05:44PM (#17269973)
LIDAR probably would have seen the obstacle and avoided it, but so would a human driver who was operating the vehicle responsibly and correctly. It sucks that people treat level 2 systems and level 3 or 4, but the same thing applies to many convenience features in a car (cruise control, power brakes, etc...). There's always going to be some bozo doing what they shouldn't be doing with something.

I'd love to see LIDAR on consumer vehicles, but AFAIK it's prohibitively expensive. And to be fair, even Level 4 autonomous vehicles still crash into things and kill people.

https://www.bloomberg.com/news/articles/2018-05-24/uber-self...

Last but not least, every semi-autonomous system all the way back to Chrysler's "AUTO-PILOT" has had similar criticisms. People in the past even said similar things about high speed highways compared to other roads WRT attention.

http://www.curbsideclassic.com/blog/history/automotive-histo...

https://books.google.com/books?id=YpGkDAAAQBAJ&pg=PA100&lpg=...

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by friendzis on Friday June 08, 2018 @03:11AM (#17263267)
> The TACC offered by most (if not all) manufacturers can't differentiate between the surroundings and stopped vehicles.

Literally every car I have driven equipped with Cruise Control and Collision Avoidance (TACC) hits the brakes and slows down to 20-ish km/h if it senses ANYTHING moving slower (including stationary) in front of the car at possible collision path.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by ploxiln on Thursday June 07, 2018 @04:09PM (#17259744)
All cars from all manufacturers are running 10s or 100s of times more code than they did 10 or 20 years ago.

https://skeptics.stackexchange.com/questions/39559/does-the-...

This really affects the nature of the situation. 20 years ago, cars contained microcontrollers with a tiny bit of code which was thoroughly reviewed and tested by skilled professionals. Today, all cars run so much code, even outside of the entertainment system, that the review and testing just can't be the same. (And there's way more programmers, so the range of skill and care is also much wider.)

We're in a new mountain-of-flaky-software world.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by ams6110 on Thursday June 07, 2018 @04:47PM (#17260104)
When the Toyota electronic throttle "unintended acceleration" accidents were in the news, the software was described as a "big bowl of spaghetti" but the NTSB ultimately determined that it was not the cause of the problems. It was drivers using the wrong pedal.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by golfer on Thursday June 07, 2018 @10:43PM (#17262172)
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by freerobby on Friday June 08, 2018 @07:27AM (#17264298)
There are news stories like this all over because it was an ongoing saga. But the DOT's investigation showed that folks were hitting the wrong pedal:

https://www.caranddriver.com/features/its-all-your-fault-the...

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by dannyfraser on Friday June 08, 2018 @04:18AM (#17263549)
Malcolm Gladwell's podcast had a good summary of the issue, including the pedal confusion.

http://revisionisthistory.com/episodes/08-blame-game

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by dekhn on Friday June 08, 2018 @09:17AM (#17264958)
I've long been curious about the "big bowl of spaghetti" comment (and all the other criticisms made by the experts who inspected Toyota's code). There were some extremely serious accusations which don't seem consistent with the fact that the vast majority of Toyotas on the road aren't showing problems caused by their MCU's spaghetti code.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by mark-r on Thursday June 07, 2018 @10:49PM (#17262195)
AI takes it to a whole new level. Neural networks are all black box, they can't be reviewed. You feed in your training data, you test it against your test data, and just have faith that it will respond appropriately to a pattern that's slightly different than anything it's seen before. Sometimes the results are surprising.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by carlmr on Friday June 08, 2018 @12:38AM (#17262626)
That's my biggest problem with AI and neural networks. You can't really measure progress here. If you wanted the same safety standards as for every other automotive software you'd have to test drive for hundreds of thousands of kilometres after every change of parameters, because there's no way to know what has changed about the AI's behavior except for testing it thoroughly.

Compare this to classic engineering where you know the changes you've made, so you can rerun your unit tests, rerun your integration tests, check your change in the vehicle and be reasonably sure that what you changed is actually what you wanted.

The other approach to autonomous driving is to slowly and progressively engineer more and more autonomous systems where you can be reasonably sure to not have regressions. Or at least to contain your neural networks to very very specific tasks (object recognition, which they're good at), where you can always add more to your test data to be reasonably sure you don't have a regression.

I don't think we'll see too many cars being controlled by neural networks entirely, unless there's some huge advancement here. Most of the reason we see more neural networks now is that our computing power has reached the ability to train sufficiently complex NNs for useful tasks. Not because the math behind it advanced that much since the 60s.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by sokoloff on Friday June 08, 2018 @09:26AM (#17265034)
> There is no evidence that said OTA update made Tesla cars any safer.

That particular OTA update significantly shortened braking distances. [The update] cut the vehicle’s 60 mph stopping distance a whole 19 feet, to 133, about average for a luxury compact sedan. That's a safer condition, IMO, and I'm uncertain how to argue that it doesn't make the car safer.

[0] - https://www.wired.com/story/tesla-model3-braking-software-up...

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by LoSboccacc on Friday June 08, 2018 @03:00AM (#17263227)
I agree with everything you said except this:

> being able to know exactly how your car will react in all situations

If one depends on intimate knowledge of his own car for safety then he’s likely already driving outside the safety envelope of the code, which was written to provide enough safety margin for people driving bad cars from 40yr ago.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by slg on Thursday June 07, 2018 @02:12PM (#17258544)
I didn't say no car has ever relied on software. I said cars are becoming more reliant on software. I don't think that is a controversial statement. I also don't think it is controversial to say that other automakers also occasionally ship buggy code. The Toyota brake issue I mentioned in the previous post is one example.

Additionally, the argument that we should continue to handle updates this way simply because we have done it this way for decades is the laziest possible reasoning. It is frankly surprising to see that argument on HN of all places.

As for the evidence that OTA updates can make things safer, this is from Consumer Reports:

>Consumer Reports now recommends the Tesla Model 3, after our testers found that a recent over-the-air (OTA) update improved the car’s braking distance by almost 20 feet. [1]

That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact.

[1] - https://www.consumerreports.org/car-safety/tesla-model-3-get...

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by txcwpalpha on Thursday June 07, 2018 @02:15PM (#17258580)
> That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact.

There is again no evidence to support this fact. There is evidence that Tesla's OTA software updates have introduced safety issues with Tesla cars. That's a fact.

Better braking distance is of course a good thing but if anything, the fact that Teslas were on the road for so long with a sub-par braking distance is more evidence of a problem with Tesla than it is evidence of a benefit of OTA updates.

The other factor in that brake story is that it took mere days for Tesla to release an update to "fix" the brakes. This isn't a good thing. The fact that it was accomplished so quickly means that the OTA update was likely not tested very well. It also means that the issue was easy to fix, which calls into question why it wasn't fixed before. It also highlights the fact that Tesla, for some reason, failed to do the most basic testing on their own cars for braking distance. Comparing the braking distance of their cars should have been one of the very first things they did before even selling the cars, but apparently it took a third party to do that before Tesla was even aware of the issue. This doesn't inspire confidence in Tesla cars at all.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by slg on Thursday June 07, 2018 @02:18PM (#17258615)
I simply don't know what to say to you if you are going to legitimately argue that shaving 20 feet off of braking distance will not make a car any safer.

EDIT: The comment I was replying to was heavily edited after I responded. It originally said something along the lines of improving braking distance is good but there is no evidence that it would improve safety.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by Lazare on Thursday June 07, 2018 @07:33PM (#17261310)
I think you're misunderstanding the question.

> if you are going to legitimately argue that shaving 20 feet off of braking distance will not make a car any safer.

Nobody is arguing that. We're arguing that there is no evidence the Tesla OTA update made the cars safer on net.

You're trying to set up some sort of "OTA updates are dangerous in general, but this one is clearly good, how do we balance it" conversation, but the problem is, this OTA update is not clearly good. OTA updates are dangerous in general, and also in this case in specific. You need to find a better example where there's actual difficult tradeoffs being made, and not just a manufacturer mishandling things.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by rad_gruchalski on Friday June 08, 2018 @05:40AM (#17263902)
> I simply don't know what to say to you if you are going to legitimately argue that shaving 20 feet off of braking distance will not make a car any safer.

If the car can’t see the obstacle, the braking distance simply does not matter.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by txcwpalpha on Thursday June 07, 2018 @02:24PM (#17258664)
And yet again, the same OTA update changed other parameters about the way the car drives that do make it less safe. I don't know why you're trying to ignore that fact. If I drastically improve the braking distance of a car, but in the same update I also make it so that the car crashes itself into a wall and kills you, is the car safer? Hint: no

As for your edit, you clearly misread the original comment, which is why I edited it for you. I said that there was no evidence that the OTA made the car safer. Please try to read with better comprehension instead of trying to misrepresent my comments.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by hfdgiutdryg on Friday June 08, 2018 @05:59AM (#17263992)
If I drastically improve the braking distance of a car, but in the same update I also make it so that the car crashes itself into a wall and kills you, is the car safer? Hint: no

You don't have enough information to come to that conclusion.

It's quite common to have to brake hard to avoid a cousin. It's pretty uncommon to see the specific scenario triggering this crash behavior.

Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by slg on Thursday June 07, 2018 @02:35PM (#17258778)
I never denied that. Your comment pointed out a problem with OTA updates and I agreed calling it "a clear flaw". I pointed out a benefit of OTA updates then asked an open ended question about how they should be handled. You responded be attacking the example I provided. I was looking to debate this serious issue, not getting into a pissing match about it.
Re:NTSB: Autopilot steered Tesla car toward traffic barrier ... by txcwpalpha on Thursday June 07, 2018 @02:37PM (#17258809)
I never said you denied it, I said you ignored it. If you wanted to debate this serious issue, then maybe you shouldn't keep ignoring one of the crucial cornerstones of the discussion. If you're unwilling to discuss points that challenge your own opinion, then it's clear that you're just trying to push an agenda rather than have an actual discussion.