News for hackers. Stuff that matters.
Re:Uber disabled Volvo SUV's safety system before fatality by romwell on Tuesday March 27, 2018 @12:42AM (#16685303)
>having a second system running that can't coordinate with the self-driving system

... is called redundancy and is the basis of building mission-critical, safe systems.

Re:Uber disabled Volvo SUV's safety system before fatality by kelnos on Tuesday April 03, 2018 @08:00PM (#16750643)
Redundancy is when the primary system fails and the backup system takes over. Since the primary and backup in this case are unrelated, created by different entities, and the entity building the primary system may not be able to fully understand conditions that will cause the backup system to engage, this isn't so much a "backup" as a "second system that could interfere in unpredictable ways".
Re:Uber disabled Volvo SUV's safety system before fatality by TomMarius on Tuesday March 27, 2018 @03:29AM (#16686146)
No, redundancy means making a backup of your primary system so it can take over, not having a completely different uncooperative system active at the same time.
Re:Uber disabled Volvo SUV's safety system before fatality by romwell on Tuesday March 27, 2018 @03:45AM (#16686238)
This is just one of many kinds of redundancy. Please see [1] for several ways in which redundancy is manifested (e.g. triple modular redundancy).

Specifically, uncooperative systems running at the same time was exactly how fault tolerance was achieved on the space shuttle[2]:

>For true redundancy to take place, all computers must listen to all traffic on all buses, even though they might be commanding just a few. That way they know about all the data generated in the current phase. They must also be processing that data at the same time the other computers do. If there is a failure [...], since the Shuttle's other three computers are sending apparently correct commands to their actuators, the failed computer's commands are physically out-voted

That's just one example. There's no need for the definition of the word "redundancy" to be as narrow as you said.

[1]https://en.wikipedia.org/wiki/Redundancy_(engineering)

[2]https://history.nasa.gov/computers/Ch4-4.html

Re:Uber disabled Volvo SUV's safety system before fatality by TomMarius on Tuesday March 27, 2018 @03:47AM (#16686247)
Redundancy definitely does not mean running two different conflicting systems. Three (identical) - that's a different story.
Re:Uber disabled Volvo SUV's safety system before fatality by InternetOfStuff on Tuesday March 27, 2018 @05:59AM (#16686834)
But they aren't conflicting: they have the same goal (not running into stuff).

In fact, there's yet another different, redundant, potentially conflicting safety system: the human safety driver.

If only one were to trigger, it still wouldn't be a conflict.

And yeah, redundancy through using non-identical systems is very much a thing, and considered much safer than having two identical things (which might make the same mistakes).

Source: I've worked on ASIL D safety-critical automotive systems, where diverse redundant subsystems were actually a design criterion.

Re:Uber disabled Volvo SUV's safety system before fatality by simion314 on Tuesday March 27, 2018 @04:26AM (#16686419)
Why not, I have 2 different systems that detect an imminent impact in a wall, then if the systems do not agree you stop, if your main system is experimental then having the backup system that is using different tech then it seems obvious that is safer to stop, then analyze the cause of the conflict.
Re:Uber disabled Volvo SUV's safety system before fatality by slowmotiony on Tuesday March 27, 2018 @05:03AM (#16686585)
So if one of these fails, the car stops? That doesn't seem like redundancy.
Re:Uber disabled Volvo SUV's safety system before fatality by tialaramex on Tuesday March 27, 2018 @06:48AM (#16687046)
Yes, it's redundantly _safe_, which is the thing we actually care about. Safety is job #1. Well, not at Uber apparently.

There are trade-offs here, sometimes we might decide to take a very small risk, after weighing it up. A famous example is ETOPS. Unlike propeller engines, jet engines for airliners are ridiculously reliable once running. ETOPS (jokingly "Engines Turn Or Passengers Swim") is a scheme which over the past few decades has gradually allowed twin engine jets (which obviously just become over-engineered gliders if both their engines fail) to fly over oceans and other large distances with no airports, on the rationale that although total engine failure would probably be deadly without anywhere to put the plane down, two unrelated sequential engine failures are now so unlikely as to be overall negligible risk to the passengers.

But that needs a risk we understand. If you have no idea what the risk is, you can't make that trade, so redundantly safe systems are the obvious choice.

Re:Uber disabled Volvo SUV's safety system before fatality by cdancette on Tuesday March 27, 2018 @05:35AM (#16686727)
The question is not about failing but detecting an obstacle and braking.

If one system detects an obstacle, stop the car. Even if this system was failing and the other wasn't detecting anything.

And if one system fail, yes the car should stop if it's not safe to drive with only one of them.

Re:Uber disabled Volvo SUV's safety system before fatality by jschwartzi on Tuesday March 27, 2018 @06:34AM (#16686982)
In engineering we don't rely on our feeling. We have to actually have reason and evidence. You're approaching it as if the car should be moving at all costs, but the responsible engineering task is to keep the car moving unless the automated systems indicate that it's hazardous to others.
Re:Uber disabled Volvo SUV's safety system before fatality by tux1968 on Tuesday March 27, 2018 @05:32AM (#16686705)
If you walk into a store and don't see that it is being robbed at gunpoint, it would be very nice to hear the assailant ordering the cashier to empty the til. You have two chances to steer clear.

Either your eyes or ears may fail you now and again, but you're better off with both.

No?

Re:Uber disabled Volvo SUV's safety system before fatality by jakeogh on Tuesday March 27, 2018 @05:48AM (#16686789)
Nothing wrong with having a chance to steer clear, but it strikes me as an odd reaction unless you live in a society where only the criminals can defend themselves.
Re:Uber disabled Volvo SUV's safety system before fatality by m_eiman on Tuesday March 27, 2018 @05:38AM (#16686739)
Seems like a way to fail safe, which is what you should want.
Re:Uber disabled Volvo SUV's safety system before fatality by jakeogh on Tuesday March 27, 2018 @06:07AM (#16686867)
unless the obstacle appears half way through an intersection where only one direction stops?
Re:Uber disabled Volvo SUV's safety system before fatality by simion314 on Tuesday March 27, 2018 @08:38AM (#16687678)
Did you skip the part where one of the system detects hitting a wall? Why is not the normal thing to slow down and stop? You will trust the experimental system that is in testing over a safer system?
Re:Uber disabled Volvo SUV's safety system before fatality by lambda on Tuesday March 27, 2018 @10:21AM (#16688638)
It's called a failsafe. The failure mode should be the safest option; which is for the car to stop or slow until the human driver takes over.
Re:Uber disabled Volvo SUV's safety system before fatality by JorgeGT on Tuesday March 27, 2018 @05:53AM (#16686809)
The human safety driver IS a completely different uncooperative system active at the same time.
Re:Uber disabled Volvo SUV's safety system before fatality by jschwartzi on Tuesday March 27, 2018 @06:31AM (#16686965)
No, in safety-critical systems redundancy means having a backup system or systems that can support the safety-critical aspects of the system if the main control system fails.