News for hackers. Stuff that matters.
Re:Self-driving vehicles against human drivers: Equal safety... by HALtheWise on Wednesday December 30, 2020 @12:24PM (#25583601)
It strikes me that a useful analogy here is the adoption of automatic elevators in buildings. In some ways, it's amazing that pretty much everyone in industrialized countries is OK with being locked in a windowless box controlled entirely by a computer, hanging over a hundreds of foot deep shaft, and in fact many people were terrified of that when Elevator operators were first replaced with computers. Some places even had operators employed to simply stand there and push the buttons to provide confidence that an trained expert was there, even though they didn't actually contribute to safety. Eventually, autonomous elevators got common enough that people will look at you really funny if you're not willing to ride in one, even though they are still responsible for ~20-30 deaths per year.
Re:Self-driving vehicles against human drivers: Equal safety... by ggreer on Wednesday December 30, 2020 @02:35PM (#25585122)
Can you provide more info about people being terrified of automatic elevators? I searched around and everything I found seems to cite one NPR article from 2015.[1] The interviewee's book is out of print and costs $100[2], so that's where that trail stops. If public sentiment against automatic elevators was as strong as described, it seems like there would be more historical evidence available. It's easier for me to find articles disparaging self-checkout systems than for automated elevators. I realize the change in elevators happened long ago, making articles harder to find, but you'd think at least one of them would have gotten digitized and indexed.

1. https://www.npr.org/2015/07/31/427990392/remembering-when-dr...

2. https://www.amazon.com/Ascending-Rooms-Express-Elevators-Pas...

Re:Self-driving vehicles against human drivers: Equal safety... by HALtheWise on Wednesday December 30, 2020 @05:27PM (#25586951)
I just spent some time digging, but finding original sources from the ~1950s is really hard without a NYTimes subscription.

These sorts of things look like promising primary sources, but I can't access the full text. https://www.nytimes.com/1949/01/12/archives/city-gets-elevat... https://www.nytimes.com/1949/02/03/archives/tenants-want-a-d... https://www.nytimes.com/1928/11/11/archives/elevator-law-cha...

Re:Self-driving vehicles against human drivers: Equal safety... by SomewhatLikely on Wednesday December 30, 2020 @07:03PM (#25587895)
I have a subscription. The first two seem to be more about tenants feeling that an operator raises the quality of life. The third one talks of several deaths of children. Apparently in these cases there was a gap of 8 to ten inches between the car doors and the landing doors which accidentally trapped the children and then the car was requested from someone on another level. An ordinance had been proposed to require operators, but opposition claims that rules requiring doors to be flush will prevent such cases.

So this would likely be in the category of cases that would probably never happen with a manual operator. Two solutions were proposed: keep manual operators, or improve the safety of the automatic system.

Re:Self-driving vehicles against human drivers: Equal safety... by tialaramex on Thursday December 31, 2020 @01:14AM (#25590078)
> Two solutions were proposed: keep manual operators, or improve the safety of the automatic system.

After Southall [1] the drivers union felt the correct approach was to double man trains. It is surely not a coincidence this would also have meant more members and thus more power and income for the union.

That accident would have been prevented by ATP (fitted to the train but disabled because the driver had not been trained) or AWS (fitted to all British trains but disabled by the driver due to a fault).

The ATP system would have braked to a halt to avoid entering an occupied track section. AWS would have audibly alerted the driver to the signals (showing a Preliminary Caution, an ordinary Caution and then Danger) and if the driver did not react, braked to a halt. With neither it appears the driver simply did not see the signals because they weren't looking and so did not react until far too late.

The correct fix of course is to require trains to have working AWS/ ATP and use it. Operating trains with more humans is a worthwhile work around to get a faulty train back to a repair yard as empty stock, but makes no sense in passenger service.

[1] https://en.wikipedia.org/wiki/Southall_rail_crash

Re:Self-driving vehicles against human drivers: Equal safety... by Guthur on Wednesday December 30, 2020 @08:01PM (#25588337)
The analogy with elevators largely doesn't work. In the main the only individuals at risk from an elevator accident are those that chose to enter the elevator, this is vastly different for automotive accidents where the individuals affecting by the autonomous car are not those that chose to use one.
Re:Self-driving vehicles against human drivers: Equal safety... by aeternum on Wednesday December 30, 2020 @08:14PM (#25588418)
If anything, that should be more of an argument for autonomous cars being allowed once they reach only 1x of the safety of human-driven-vehicles.

Almost by definition, more pedestrians will die if we wait for the 4-5x safety threshold proposed in the article.

Re:Self-driving vehicles against human drivers: Equal safety... by studius on Wednesday December 30, 2020 @09:04PM (#25588757)
The 4-5x is based on statistics of a sample with a high ratio of human drivers to autonomous drivers, if done on the road today.

Autonomous won't be known to be N times as safe as human driving until there are sufficient autonomous cars on the road and the varying ratios of autonomous to human drivers have been at equilibrium for a some sufficient amount of time to gather sufficient data to compare to human driving safety statistics.

So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet.

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @09:47PM (#25589048)
> So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet.

"could be"? Statistics? Aren't these studies asking about theoretical cars that are by definition a certain amount safer?

Re:Self-driving vehicles against human drivers: Equal safety... by studius on Wednesday December 30, 2020 @10:02PM (#25589131)
e.g. "This autonomous car is 4-5x safer than your human-driven car."

When people hear numbers like that, they may assume it's based on sufficient data and is true, even if it's theory.

But how could it be true or acceptable, given that we're only just starting to have more autonomous vehicles on the road, and the safety could be dependent on the ratio of autonomous vehicles to human drivers?

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @10:09PM (#25589168)
Are you talking about the study, or people claiming that about real cars? The study would need to have a wide variety of safety numbers, and all the cars are theoretical. If 4-5x lines up with a claim about a real car, it's probably a coincidence. Saying there's not enough data about real cars has no bearing on this research at all, unless the abstract is wildly misleading.

You replied to someone talking about waiting for autonomous cars to be 4-5x as safe. That means they are waiting for a car to meet that threshold with correct statistics. You can't call out nonexistent statistics as being a lie!

Re:Self-driving vehicles against human drivers: Equal safety... by studius on Wednesday December 30, 2020 @10:16PM (#25589212)
We have safety statistics on real cars with a minority of autonomous vehicles on the road.

What we don't have are adequate safety statistics on autonomous cars on the road with each other and human drivers at various ratios of autonomous cars to human-driven cars.

If a study were to be able to propose safety numbers based on the various ratios and various autonomous cars and systems, then that would be adequate, given the theories behind those numbers are well-founded.

If you were to say, "This is how it is: given that the ratio of autonomous cars to human-driven cars isn't changing and we expect it to stay the same, we can see that the autonomous cars are 4-5x safer," then I'd not call it a lie.

But 4-5x safer without qualifications shows a gross misunderstanding of the importance of the effect of the ratio of autonomous drivers to human drivers; the variation in environment and expected behavior could influence greater or decreased safety, leading false assumption.

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @10:42PM (#25589349)
It would be easy to do a comparison with different ratios, though, so you shouldn't preemptively assume incompetence.

And it seems pretty unlikely that increasing the number of autonomous cars will make them signficantly less safe, and that's the only result that would be a problem. Mild fluctuations don't matter on a scale as coarse as "4-5x", and an improvement would be good.

Re:Self-driving vehicles against human drivers: Equal safety... by kaba0 on Thursday December 31, 2020 @06:50AM (#25591836)
Do you think that increasing the number of autonomous cars will linearly decrease the number of accidents to a certain point? Don’t you think that increasing their number will increase the chances of meeting with really bad human drivers and we simply don’t have sufficient info on whether those ‘meetings’ are less or more deadly than with a human driver - and those are the cause of a significant chunk of accidents. And by intuition I doubt that today’s AI could react better than a competent human driver to someone cutting in front of it, and the like. Simply because we are better at reading high-level patterns in others’ driving. Reaction time is not the only metric that matters.
Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Thursday December 31, 2020 @11:11AM (#25593929)
> Do you think that increasing the number of autonomous cars will linearly decrease the number of accidents to a certain point? Don’t you think that increasing their number will increase the chances of meeting with really bad human drivers and we simply don’t have sufficient info on whether those ‘meetings’ are less or more deadly than with a human driver - and those are the cause of a significant chunk of accidents.

There's a difference between the risk changing, versus merely going from insufficient data to sufficient data.

When you have an extremely small data pool, it's also quite possible that one or two meetings with a really bad driver will give you a misleadingly bad impression of autonomous cars.

But I'll put it this way. Once we've seen either 10 billion miles or 100 fatalities from a particular tier of self-driving, we'll have a very solid idea of how dangerous it is. Getting that much data only requires a tenth of a percent of cars in the US for three years. (And if they're particularly dangerous we can easily abort the test early.)

> And by intuition I doubt that today’s AI could react better than a competent human driver to someone cutting in front of it, and the like. Simply because we are better at reading high-level patterns in others’ driving. Reaction time is not the only metric that matters.

If someone's dangerously cutting people off there probably isn't much to read in their patterns. Being cut off seems to me like one of the situations that is most about reaction time and least about thinking.

Re:Self-driving vehicles against human drivers: Equal safety... by floatingatoll on Wednesday December 30, 2020 @08:33PM (#25588547)
While "those unable to stairs up to floor XYZ" can be proven to be a statistically smaller subgroup than "all possible elevator riders", this argument may not hold much water. If you're a wheelchair user or other less-abled person, and your choice is between stairs and an elevator of any sort, you weren't really given much of a choice in that matter.
Re:Self-driving vehicles against human drivers: Equal safety... by kaba0 on Thursday December 31, 2020 @06:42AM (#25591783)
I know it’s an analogy, but elevators are dead-simple concepts with many fail-safety mechanisms independent of the controlling ‘program’. Self-driving cars are ridiculously complicated black-box program with at most software fail-safes that are itself more complicated than one can reason about so the only thing remaining is empirical testing in real-world situations. And every single bugfix have to be retested substantially as it can just as easily make something else worse - and I don’t feel confident about today’s “self-driving” cars
Re:Self-driving vehicles against human drivers: Equal safety... by dekervin on Wednesday December 30, 2020 @12:46PM (#25583864)
there is some kind of implied machine capability that a layman assign to a computer. If a task is culturally ( through movies , series, ... ) though to be within that implied capability people will be comfortable. ( cf automatic trains, ... )
Re:Self-driving vehicles against human drivers: Equal safety... by RcouF1uZ4gsC on Wednesday December 30, 2020 @05:49PM (#25587194)
The difference with elevators is that the safety systems are actually in large parts independent of the control. Since the Otis safety elevator, you could go so far as to cut the elevator cord and it would still be ok.

With self-driving cars, you don't have those type of backup safety systems.

Re:Self-driving vehicles against human drivers: Equal safety... by a_t48 on Wednesday December 30, 2020 @06:33PM (#25587627)
You don't have those sorts of systems for regular human drivers, either. If a human driver passes out (diabetic episode, heart attack, drunkeness), you are just as SOL. With a self driving system there's various levels of redundancy you can put in, up to some sort of dead man's switch on an embedded controller that slams the brakes (or maybe just a gentle stop in combination with existing ADAS) if all of the "higher level" systems stop responding.
Re:Self-driving vehicles against human drivers: Equal safety... by paulryanrogers on Wednesday December 30, 2020 @07:03PM (#25587891)
It's true that's a similar problem with humans. Yet when it occurs to the same person often they may face restrictions on their driving privileges.

When AI goes bad it could impact whole fleets simultaneously. And for all the many ways we know software can unintentionally go bad the liability can be a rabbit hole.

Re:Self-driving vehicles against human drivers: Equal safety... by ageofwant on Wednesday December 30, 2020 @07:39PM (#25588169)
That seems very unlikely. 'AI' driving systems are subjected to millions of hours of simulated driving. A very good human driver will have a few thousand hours of experience after decades of driving.
Re:Self-driving vehicles against human drivers: Equal safety... by chrisco255 on Wednesday December 30, 2020 @08:23PM (#25588472)
Has your computer or phone ever glitched? Have you ever suffered a Malware attack? Every single way that computers are vulnerable, machine driven systems are vulnerable. We just saw the SolarWinds hack that was installed through malicious software updates. That could happen in an autonomous driving system as well.

Humans are still far better at dealing with unforeseen circumstances than computers. An AI is trained on a dataset and can overfit for certain parameters.

Re:Self-driving vehicles against human drivers: Equal safety... by ageofwant on Wednesday December 30, 2020 @09:18PM (#25588861)
Releasing unverified AV code would breach most every 'duty of care' legislation in existence. Releasing it in a way that compromises thousands of systems at a time would be the same. These are devops issues not capability.

An AV system needs to pass the driving test your 16 year old sister passed, that's it.

Re:Self-driving vehicles against human drivers: Equal safety... by chrisco255 on Wednesday December 30, 2020 @10:32PM (#25589306)
That has already happened with network security software (SolarWinds) used at the highest levels of government including the Pentagon, White House, Congress, and most of the Fortune 500. There is no reason to believe that a massive AI driving system could not be hacked in a similar way through a malicious exploit or update. In fact, adversial AI could be used to dupe these tests and achieve the effect. If 50 million cars on the road were using one particular software suite, this is a vector of attack that has to be considered. It is not an impossibility and you cannot hand wave it away.
Re:Self-driving vehicles against human drivers: Equal safety... by itchyouch on Wednesday December 30, 2020 @10:14PM (#25589196)
The current L2 system developed by Comma.ai simulates the model and changes in CI on every commit from what I can tell.
Re:Self-driving vehicles against human drivers: Equal safety... by josefx on Wednesday December 30, 2020 @08:33PM (#25588550)
The average human driver doesn't need millions of pictures of a stop sign to identify one either. Training current AIs sucks and the results need to be verified because they can be flacky, the current research really doesn't come close to match an organism that spend millions of years evolving the ability to move in a complex 3D world.
Re:Self-driving vehicles against human drivers: Equal safety... by ageofwant on Wednesday December 30, 2020 @09:26PM (#25588918)
The AV problem is not as intractable as you imagine. Driving a car is not the same as dating a girl.
Re:Self-driving vehicles against human drivers: Equal safety... by paulryanrogers on Wednesday December 30, 2020 @07:46PM (#25588214)
Which seems unlikely?

Do AI drivers face the same variety of conditions as humans? Or is it millions of hours on near [perfect] roads in a temperate climate?

Re:Self-driving vehicles against human drivers: Equal safety... by tsimionescu on Thursday December 31, 2020 @01:54AM (#25590266)
You can't compare the times, because AI and humans "learn" in vastly different ways, at vastly different rates.
Re:Self-driving vehicles against human drivers: Equal safety... by kaba0 on Thursday December 31, 2020 @06:56AM (#25591867)
Can an AI system learn on the fly (that would negate the testing done by the way)? Because I’m fairly sure other than sensor input “normalization”, they can’t and humans can.
Re:Self-driving vehicles against human drivers: Equal safety... by tomp on Wednesday December 30, 2020 @06:57PM (#25587838)
It's in a human's interest to not kill him/herself.

The interests aren't aligned at all with self-driving cars. Do you think any executives, engineers or salespeople will go to jail if anyone dies? Think again.

Re:Self-driving vehicles against human drivers: Equal safety... by Fricken on Wednesday December 30, 2020 @07:17PM (#25587995)
Humans typically aren't interested in killing themselves, but their judgement can also be pretty bad. 40% of traffic fatalities involve impaired driving. In 2018 Distracted driving was attributed to 2,800 fatalities.

Uber ATG's fatal pedestrian collision was the beginning of the end for the program. Uber CEO Travis Kalanick was pushed out of the company, the program has been shuttered, and one of the top engineers, Anthony Levandowski, is in jail (although Levandowski was not implicated in the accident, he's also not invulnerable).

Re:Self-driving vehicles against human drivers: Equal safety... by morelisp on Wednesday December 30, 2020 @08:24PM (#25588477)
You think Kalanick was pushed out because of the one self-driving crash, and not the sexual harassment scandals, theft of medical records, or threats to journalists?

(Of course it wasn't any of those - it was because the self-driving program wasn't anywhere close to working even ignoring all the safety problems. You can do all those things - the only thing you can't do is waste investor money.)

Re:Self-driving vehicles against human drivers: Equal safety... by Fricken on Wednesday December 30, 2020 @08:38PM (#25588580)
Travis autonomy would take a couple years. He spent some absurd sum acquiring the talent for ATG and gave them unrealistic deadlines. The program was a shit show. A pedestrian was killed, the program was stopped, Investor money was lost. While this is not the only factor leading to Kalanick's ousting, it was a big one.
Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @09:29PM (#25588933)
The problem isn't total system failure, it's mistakes.

A mistake in a car can cause a crash. A mistake in an elevator can't cause a crash.

Re:Self-driving vehicles against human drivers: Equal safety... by hypertele-Xii on Wednesday December 30, 2020 @06:53PM (#25587806)
The problem is, what if the higher level systems are responding, but are confused by the data they see? We'd have to have redundant decision-making like in the space industry - two of each computer and sensor that have to agree on the result, otherwise safety systems stop the vehicle.
Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Wednesday December 30, 2020 @08:45PM (#25588638)
We're going to end up at the trolley problem again. "Should I do action X and kill 3 or do action Y and maybe kill 2?"
Re:Self-driving vehicles against human drivers: Equal safety... by LinuxBender on Thursday December 31, 2020 @10:34AM (#25593544)
This was the scenario in the depressing flashbacks in the movie iRobot. [1] A cop, truck driver and a passenger car with a little girl were pinned together under water in a lake. The NS4 (robot) passing by saw the accident and calculated the cop would have the highest probability for survival and rescued him and not the child, giving him PTSD.

I am morbidly curious how much from that movie will play out with cars and other machinery as things evolve. I am also curious if we will learn any lessons ahead of time or if like city traffic lights, {n} number of people have to die before it becomes a financially viable discussion.

[1] - https://www.youtube.com/watch?v=_MFGx8d1zl0

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @10:14PM (#25589197)
Don't waste too much time on that. These are cars. They can just brake, it's good enough.
Re:Self-driving vehicles against human drivers: Equal safety... by troupe on Wednesday December 30, 2020 @10:56PM (#25589425)
If a self driving car hits a school bus at high speed (despite braking), I'm pretty sure a lawyer will make a case that it could have avoided the school bus by driving up on the side walk which may or may not have had people on it.

I get your point, but I think it is probably more correct for people who are making a split second decision rather than for a car where the decision has already been made in code and someone (or some company) has to take responsibility for why it was made that way.

Re:Self-driving vehicles against human drivers: Equal safety... by kaba0 on Thursday December 31, 2020 @07:02AM (#25591894)
Today’s AIs are happy to correctly identify which lane they should be on - counting how many people are on a given vehicle is so way out of scope that there is no point continuing.
Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Thursday December 31, 2020 @02:03PM (#25595886)
This is a strawman.

If the option is available to avoid collision with another car and it brakes instead, colliding and killing someone else, that will be a lawsuit and that lawsuit will easily win.

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Thursday December 31, 2020 @12:19AM (#25589845)
You'll have lawyers willing to make a case against you even if you make a provably perfect choice.

And the real issue is whatever got you into that situation, not the split second choice you make.

Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Wednesday December 30, 2020 @10:16PM (#25589211)
Let me know when we reach the point that autonomous cars can bend the laws of physics too, that would truly be a hoot!
Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Wednesday December 30, 2020 @10:31PM (#25589298)
I think you deeply misunderstood me.

I'm not saying that braking is magic and solves all problems. I'm saying that braking is morally sufficient in the real world. If hitting something is unavoidable, you can hit whatever is directly in front of you while braking as hard as possible. It's good enough.

Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Thursday December 31, 2020 @10:16AM (#25593387)
That's good enough for a human.

Not for a machine which you can file a lawsuit over and say "it had a decision that it can actually make and chose to kill my son." See how that changes things? It's no longer "an accident."

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Thursday December 31, 2020 @11:12AM (#25593939)
Eh. You can say the exact same thing about a human.

Apply the reasonable person standard. Are they going to do any better? Probably not.

Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Thursday December 31, 2020 @02:02PM (#25595872)
No you can't. Humans don't have near the reaction time as these self driving cars do. They can process much more than we can, and they never get distracted, like we do.

If a self driving car kills someone and it had the choice/option to do something else then we have a problem that needs dealt with. This isn't a human being or human error.

Re:Self-driving vehicles against human drivers: Equal safety... by Dylan16807 on Thursday December 31, 2020 @03:37PM (#25596841)
Humans can panic and decide to swerve, and it only takes them a fraction of a second longer than it takes a self-driving car. Someone willing to argue that the car had a choice and made the wrong one could just as easily argue that the human had a choice and made the wrong one.

You might personally think that oh, the machine is being rational and intentional, that's different from a human. But a lot of people will treat a human the exact same way. That split second panic is judged as if they had all the time in the world to consider the optimal choice.

The legal system already has and handles this type of lawsuit.

Re:Self-driving vehicles against human drivers: Equal safety... by a_t48 on Wednesday December 30, 2020 @08:47PM (#25588650)
There's nothing stopping someone from making a self driving car with two compute systems and multiple types of sensors with overlapping FoVs. In fact, you can usually see on most self driving cars - a minimum of two roof mounted LIDAR (which share FoV in front and back) and various cameras.
Re:Self-driving vehicles against human drivers: Equal safety... by ntsplnkv2 on Wednesday December 30, 2020 @08:43PM (#25588623)
I don't think a self driving car that has to make rapid decisions in quickly changing circumstances with tons of moving parts outside of its control is comparable to a specific use case with much fewer variables like an elevator.
Re:Self-driving vehicles against human drivers: Equal safety... by RhodesianHunter on Wednesday December 30, 2020 @09:20PM (#25588877)
Why not? The relative complexity of an automatic elevator at the time considering their level of tech was probably comparable to the complexity of an autonomous vehicle today.
Re:Self-driving vehicles against human drivers: Equal safety... by kaba0 on Thursday December 31, 2020 @06:54AM (#25591852)
Lol. At the level of tech? We are barely better than a lego mindstorm that goes forward until something is not closer than x centimeters.. (with a bit of sarcasm)

An autonomous car can’t really exit a parking building (I’m not really sure what are they called, but those multi-level parking thingies) because it would require them to understand signs and the like and those may often be missing.

Re:Self-driving vehicles against human drivers: Equal safety... by RhodesianHunter on Thursday December 31, 2020 @07:18PM (#25598973)
I rode around Vegas in an automated car at reinvent last year. It understood signs just fine.

they have thousands and thousands of miles of autonomous driving across California and Nevada.