On the contrary: the Volvo system is designed not to fight but to override any other input, including the human driver. In any other Volvo car currently on the road that detects an obstacle, the Volvo system will override the driver and bring the car to a stop even if the driver insists in hitting the obstacle: https://www.youtube.com/watch?v=oKoFalJiazQ

Independently-working override-capable systems are the base of engineering redundancy safety. See airborne collision avoidance systems (ACAS), which will automatically and forcefully steer an aircraft to avoid a collision if necessary: https://en.wikipedia.org/wiki/Airborne_collision_avoidance_s...

I don't have experience with self-driving cars, Uber or Volvo safety systems, but my Jeep's backup collision prevention often slams the brakes if I'm backing up and the camera detects potholes, oil stains, or irregular painted lines in the parking lot.

If any of the systems are vulnerable to any such false positives, and equipped to enable emergency braking to avoid them, even on the highway, it's not hard to imagine why they might be disabled, especially during testing phases.

I think it's fair to say that at this early stage in product development, there's probably no 'always right' answer for how to handle a given obstacle without considering locality, driving speed, road conditions, likelihood of false positives and negatives, etc.

They shouldn't be testing on public roads and highways then.

Yeah mine does that too. I wish they had a sensitivity option like they do with the forward collision sensors.

Hey there's an idea for Uber, maybe instead of disabling the forward collision system entirely they could just decrease the sensitivity to lessen false positives (like in our Jeeps)?

TCAS systems won't automatically take over. Instead they audibly issue an order to the flight crew, describing the action that must be taken, and the flight crew must take the action.

Instructions from TCAS are near absolute in their priority. If ATC says to do something different, you ignore ATC and do what TCAS says. If the pilot in command says to do something different, you ignore the pilot in command and do what TCAS says. If somehow God Himself is on your plane and tells you to do something different, you ignore Him and do what TCAS says. Compliance with TCAS is non-negotiable, and the Überlingen disaster[1] is the bloody example of why it's that way.

Self-driving/autonomous-car systems need to have a similar absolute authority built in. If Uber disabled theirs because of false positives, it's a sign Uber shouldn't be running those cars on public roads.

[1] https://en.wikipedia.org/wiki/2002_%C3%9Cberlingen_mid-air_c...

AFAIK Airbus has a TCAS-capable AP/FD, certified at least for the A380, which can automatically fly RAs by itself: https://www.skybrary.aero/index.php/Autopilot/Flight_Directo...

If the Uber engineers shared your confidence that it would play out that way, they would not have disabled it.

Redundant safety systems are a great idea, evidently Uber needed more of them, and integrating with the Volvo system might have been a reasonable option. It's silly to suggest that the integration would necessarily have been trivial, though. That's what I'm objecting to.

A woman is dead because of some combination of overconfidence, hubris, and ignorance of Uber’s self driving engineers.

The above traits aren’t exclusive to Uber’s “engineers”, but are lethal when applied to the engineering of life safety systems.

I am not sure if I would have been able to avoid such a situation myself. That's the only thing that keeps me from excoriating Uber...

How many people are still alive because Uber had the restraint to not slap control systems together willy-nilly, like so many people here seem to think is an obviously great idea?

A lot more than one, that's for sure.

Yikes. I would not allow a business to continue to exist whose argument is “at least we’re only partially wreckless”.

See how that holds up in civil court in front of a jury, or any legislative body Uber might need to convince to allow their operation in a jurisdiction in the future.

“Just think of how many more people we would’ve killed if we didn’t care at all!”

"at least we’re only partially wreckless" has been Tesla's standard operating procedure for a while now. I'm guessing you think Tesla shouldn't exist either.

Oh, they'll get crucified for it, I'm sure -- because the public doesn't understand that integration is never trivial, and that "obvious" integrations aren't always good ideas.

I had thought an audience of developers would "get it," since we deal with fallout from ill-conceived integrations every day, although admittedly in a far less spectacular form than control system engineers.

Unfortunately, the uber hate train has already left the station and there's no slowing it down until the investigation finds (or doesn't) actual evidence of negligence rather than clickbait guesswork by armchair engineers. Too bad.

> I had thought an audience of developers would "get it," since we deal with fallout from ill-conceived integrations every day, although admittedly in a far less spectacular form than control system engineers.

You do a disservice to the audience by assuming it would be understanding of grossly negligent behavior.

Failures happen, that is to be expected. If you're building self-driving vehicles, you're supposed to be engineering for those failures. Disabling two life safety systems (Volvo's AEB and Uber's own AEB) and relying on a single inattentive human driver? I don't understand how that's understandable or justifiable in any scenario besides a carefully controlled test track.

I think you’re misunderstanding. According to the report, it’s not just the Volvo emergency braking system that was disabled. Uber’s self-driving system had its own emergency braking feature that was also disabled.

How many people were not killed by Uber? That's a terrible way to look at things. It is better to wonder _why_ Uber killed a person. The answer appears to be right in the NTSB report about the brake system.

You're being very charitable in your assumption that Uber 'engineers' gave it much thought at all.

(Were any Professional Engineers even involved in this? Or was it just a bunch of "software engineers"?)

However much thought they gave it, I guarantee that everyone in this comment train has given it much less. It wasn't a trivial decision, their choice is not prima facie evidence of negligence.

Again, very charitable. Your assumptions are baseless, seemingly motivated by little more than your trust that incredible incompetence and negligence is rare in the real world.

Again, not at all charitable. Your assumptions are baseless, seemingly motivated by little more than your mistrust that in the competence of people that have been actively working on this problem, many for years.

He said that he can "guarantee" Uber engineers gave it careful consideration (or at least more than people in this thread), but in truth he can do nothing of the sort. There are a plethora of examples where engineers and "engineers" didn't give a problem careful consideration and it got people killed. There is no rational basis for guaranteeing this is not one of them.

There is no "guarantee" that uber "engineers"/engineers put more thought into this than "their system is inconvienent, tear it out" or "we don't need their system because ours will be better, tear it out." Nobody can guarantee Uber engineers were not stupendously negligent until the investigation is complete. Anybody who thinks they can guarantee that has an irrational basis for thinking they can provide such a guarantee.

You might be troubled to know that the guidance control systems on modern spacecraft and many aeronautics subsystems work this way -- at least three redundant systems have access to sensor data and each make independent decisions about the course of action, if any, needed. In short, they vote, and majority wins.

There is no "fight with each other". If any system says "break", the car breaks. If none says that, the Uber system has control.

Every time you drive a modern car, you fight for control with a computer driving system.

Somehow, it works out.

Unclear why this is being downvoted. Modern cars have all sorts of computer driving systems, e.g. ABS. If you learned to drive without it, you still think to pump even though the computer does a better job if you just slam on the brakes.

The point is that so far most (all?) of these computer systems don't have a failure mode where it kills pedestrians, because their scope is very limited.

Because one doesn't "fight" with ABS not wanting to break. It will break when you hit the brake.

Same with power steering: it won't steer into a different directorion. You don't have to "fight" with it.