News for hackers. Stuff that matters.
Re:Uber’s self-driving car could not detect pedestrians outs... by isostatic on Wednesday November 06, 2019 @08:22AM (#21462214)
It wasn't that the sensors were confused or something - the AI had no idea how to cope with pedestrians in the road for whatever reason, so threw them in the "Other" category.

At least 98% of objects are categorised, the rest don't matter. /s

Re:Uber’s self-driving car could not detect pedestrians outs... by wongarsu on Wednesday November 06, 2019 @08:36AM (#21462345)
Consistently hitting the "other" category would be a massive improvement over what happened. According to the timeline for the pedestrian was initially classified as a vehicle, later as a bicycle, with short periods in between where she's classified as "other" or "unknown". And of course when an object changes classification it loses all history and is initially predicted to be static. Despite detecting her for 5.6 seconds the system never had more than 1.5 seconds of tracking history because it couldn't make up it's mind and kept throwing everything away.
Re:Uber’s self-driving car could not detect pedestrians outs... by rvense on Wednesday November 06, 2019 @08:25AM (#21462236)
TBH this does sound exactly like something you'd expect to happen if you took some Javascript ninja used to moving fast and breaking things and asked them to build a car. But surely Uber has more sense than that...
Re:Uber’s self-driving car could not detect pedestrians outs... by dsfyu404ed on Wednesday November 06, 2019 @08:34AM (#21462324)
Coming from the defense (go ahead, clutch your pearls, I won't be offended) industry I just can't understand why people ship things that fail into in potentially dangerous states on some known non-trivial fraction of inputs. If the purpose of the software were entertainment or some other low stakes thing I'd understand but software controlling things that can kill people or lose lots of money need to be able to handle unexpected inputs gracefully.
Re:Uber’s self-driving car could not detect pedestrians outs... by Majromax on Wednesday November 06, 2019 @08:50AM (#21462512)
> I just can't understand why people ship things that fail into in potentially dangerous states on some known non-trivial fraction of inputs

Because that kind of safety-conscious culture needs to be deliberately enforced, from the top all the way down, by a system that takes a long view.

If we're making an analogy to the defense industry, then building self-driving cars as a tech startup is closer to the Anarchist's Cookbook than Northrop Grumman.

Re:Uber’s self-driving car could not detect pedestrians outs... by Eric_WVGG on Wednesday November 06, 2019 @08:40AM (#21462390)
Because the penalties and rewards don’t align.
Re:Uber’s self-driving car could not detect pedestrians outs... by targonca on Wednesday November 06, 2019 @10:09AM (#21463287)
Because you don't want to be the one whose team is lacking in agility on the next company all-hands meeting ;)
Re:Uber’s self-driving car could not detect pedestrians outs... by peteradio on Wednesday November 06, 2019 @08:36AM (#21462348)
Or managers barking at sheepish recent grads/h1bs who don't know how to or can't push back. Does not seem like a very strong engineering culture.
Re:Uber’s self-driving car could not detect pedestrians outs... by fortran77 on Wednesday November 06, 2019 @08:46AM (#21462472)
What are you saying, that a Haskell or Rust elite programmer wouldn't have made this mistake!?
Re:Uber’s self-driving car could not detect pedestrians outs... by nosianu on Wednesday November 06, 2019 @08:58AM (#21462599)
The strong types in those languages would have prevented a reference to a "Person" object to be treated like a reference to an "Other" object... /s
Re:Uber’s self-driving car could not detect pedestrians outs... by svrtknst on Wednesday November 06, 2019 @08:28AM (#21462267)
Yes... surely.
Re:Uber’s self-driving car could not detect pedestrians outs... by Gupie on Wednesday November 06, 2019 @08:28AM (#21462266)
Isn't the bigger problem that it did not classify the object as being on a collision path. It doesn't matter what the object is, brake if you are going to hit it.
Re:Uber’s self-driving car could not detect pedestrians outs... by ACS_Solver on Wednesday November 06, 2019 @09:06AM (#21462681)
It's astonishing to me that emergency brakes were off. The investigation report clearly shows that the pedestrian was classified incorrectly, alternating multiple times between wrong classifications, and the path prediction was off. Despite all of those failures, it was still correctly determined that a crash was imminent 1.2 seconds before impact, and a second later realized that avoidance had failed - at some point around 1.2 seconds before the crash, AEB should have engaged.

Instead, as the report says:

> The vehicle was factory-equipped by Volvo with several advanced driver assistance systems (ADAS), including forward collision warning (FCW) system and automatic emergency braking (AEB) system. However, Volvo collision avoidance ADAS were not active at the time of the crash; the interaction of the Volvo ADAS and ATG ADS is further explored in section 1.9.

It was apparently disabled by Uber because it was difficult to run the Volvo systems alongside their own, which strikes me as highly irresponsible. Autonomous driving software relies on multiple redundancy, and AEB is the last-resort system for when everything else, from automatic maneuvers to alerting the driver, has already failed.

Re:Uber’s self-driving car could not detect pedestrians outs... by ForHackernews on Wednesday November 06, 2019 @08:32AM (#21462300)
> It doesn't matter what the object is, brake if you are going to hit it.

The built-in emergency braking system from Volvo does this [0] but Uber deliberately disabled it (presumably because it conflicted with their self-driving rig).

[0] https://www.media.volvocars.com/global/en-gb/media/pressrele...

Re:Uber’s self-driving car could not detect pedestrians outs... by rightbyte on Wednesday November 06, 2019 @05:30PM (#21468284)
Atleast the Volvo system output could atleast be used as a sanity check or something.

If you hit the accelerator harder when the Volvo brakes you override it. It should be fairly easy to integrate as a backup.

However I guess that the Volvo system might not be far looking enough for those speeds?

Re:Uber’s self-driving car could not detect pedestrians outs... by fabian2k on Wednesday November 06, 2019 @08:31AM (#21462291)
From what I read in some of the earlier reports on this, the car didn't have the ability to emergency brake in autonomous mode. It was disabled at that time, so it could only brake in regular traffic, not for obstacles that appear suddenly.
Re:Uber’s self-driving car could not detect pedestrians outs... by mannykannot on Wednesday November 06, 2019 @08:46AM (#21462459)
That is right - so now we have two major errors.

The justification made for disabling the emergency braking - that it would interfere with data gathering - might appear reasonable at first sight, but it does not stand up to scrutiny, for if the emergency braking is triggered, the driving system has already made a mistake, and you already have the data on that malfunction.

Re:Uber’s self-driving car could not detect pedestrians outs... by kjaftaedi on Wednesday November 06, 2019 @08:57AM (#21462596)
Yes, but accidental emergency breaking can cause just as bad of an accident as this, it all just depends on the scenario.

They had hired a driver to sit behind the wheel to monitor the road and the car for exactly this reason.

The driver they hired decided to watch a movie on their phone instead of paying attention to the road.

Re:Uber’s self-driving car could not detect pedestrians outs... by mannykannot on Wednesday November 06, 2019 @09:36AM (#21462956)
> Yes, but accidental emergency breaking can cause just as bad of an accident as this, it all just depends on the scenario.

If Volvo's system is that dangerous, then it should not be on the road at all - but there is no evidence that it is, you are just making a speculative argument.

> They had hired a driver to sit behind the wheel to monitor the road and the car for exactly this reason.

That is no reason to disable a safety feature that would add safety in depth.

> The driver they hired decided to watch a movie on their phone instead of paying attention to the road.

That was a major error - a crime, in fact - but, unfortunately, also an entirely predictable scenario that cannot be dismissed on the grounds that dealing with it would make testing more difficult or expensive. So now we have three errors.

Re:Uber’s self-driving car could not detect pedestrians outs... by Piskvorrr on Wednesday November 06, 2019 @10:56AM (#21463763)
Does not compute. I regularly hit objects while driving my car. Such as: potholes, twigs, paper bags, plastic bottles.

The deadly assumption here is "doesn't clearly match a category" => "safe to ignore"

Re:Uber’s self-driving car could not detect pedestrians outs... by peteradio on Wednesday November 06, 2019 @08:34AM (#21462318)
Emergency brakes were disabled because they behaved erratically. So I'm not sure what it would have done if it had ID'd her correctly, it was still relying on witless behind the wheel. What a shit show.
Re:Uber’s self-driving car could not detect pedestrians outs... by mannykannot on Wednesday November 06, 2019 @08:52AM (#21462541)
I doubt that the emergency braking system was behaving erratically in itself, to any significant extent - if that were the case, it would do so for human drivers, and so should not be on the road at all. What I suspect is more likely is that Uber's system was behaving erratically, triggering the emergency response with some frequency.
Re:Uber’s self-driving car could not detect pedestrians outs... by peteradio on Wednesday November 06, 2019 @09:26AM (#21462862)
Yea, should have put "behaving erratically" in quotes, I think you were probably on the money that it was in response to the AI or AD in this case. Probably was behaving like antilock brakes as the classification flipped every few milliseconds.