Further, we don't stop using trains, or buses when a pedestrian makes an error and gets killed because they "could have not been hit, maybe". For those who have paid attention to those other accidents(even though the local transit isn't as sexy to hate on as Uber, and they didn't involve AI), it can actually be quite shocking how little of a blip it causes in the transportation system.
I'm confident that self-driving cars will eventually be safer than human driven ones. I'm not confident that this can be realistically achieved without killing some unfortunate bystanders in the process of getting to that point. That doesn't mean we should accept these cars running amok killing people all over the place. It doesn't mean I think this specific case involving Uber is "reasonable" either. I just think, in a general sense, a realist has to be able to try to find some balance between short term lives lost and long term lives saved.
Even drug companies can't know what they don't know and end up killing people with extensively tested drugs. Airplanes fall out of the sky because of unknown bugs that simply cannot be reasonably tested for.
Human progress relies on taking mitigated risks.
[1] http://www.businessinsider.com/ubers-fake-city-pittsburgh-se...
Medical industry is heavily regulated now, but regulations appeared after it has grown in size. Similarly transport, construction, and anything else I could think of.
The reason it happens this way is that it's hard to set up standards and practices for something that is entirely new - when you set them up too soon, they may end up impossible to achieve, and block innovation altogether.
(Similarly, and less comically in retrospect, crash testing regimes resulted in a lot of safety improvements that would have been unlikely as a result of purely commercial considerations)
The 14mph and no red flag escort was only for machines <3 tonnes, in 1896, probably only a few 100 machines on the road then anyway
The issue here is that almost every industry began in times when we collectively didn't give much shit until things went wrong. If you wanted to make a watch factory using radium - sure, go ahead. It was only after people started dying that we figured out that maybe this needs regulation. But if you wanted to make a commercial product in 2018 using radium you would have to go through plenty of hoops, and rightfully so.
I don't see why autonomous cars are any different - I don't see why we're starting with the assumption that it's fine to test this without strict regulation applied to everything else automotive. Like, if you wanted to start a startup making new tyres using a revolutionary compound, it would not be allowed to just go ahead on public roads - they would need to pass extremely strict testing that we apply to tyres since they are a life/death kind of component. But autonomous cars are allowed as-is right now? With a failure mode where the car literally doesn't react at all to a person crossing the road, despite having 4(!) sets of sensors used to detect this exact kind of obstacle? That is an engineering failure of the worst kind. The law tried to make it better by requiring a driver behind the wheel - but that's still clearly not enough.
It was actually Peter Thiel's grief against over regulation. It kills creativity in domains that are much more important to mankind than websites.
Is version 1.0 of a product technically considered patched? I'm assuming, categorically, no..
That said, despite where we're talking about it, Uber is not primarily in the tech industry - they don't sell you software, they sell you rides - so they are a taxi business, foremost. The taxi business (artificially chauffeured, or, not,) is actually a perfect example of an industry where regulation makes it palatable at all, compared to what v1 taxi cabs were like...
There is also the issue of whether or not we even have the means to regulate (or even come up with a formalization for the axiomatic ethical examination of near-black-box/difficultly analyzable tech).
That's a separate concern from whether or not the consumer will be liable for mistakes made by the provider until regulation is in place.
imo, innovation is pretty unimportant compared to regulation, and far less valuable, too..
If stuff like this continues to happen, i can see some government banning self driving tech for the foreseeable future.
Even though in absolute terms there are more deaths by automobile, that is also because people are typically around cars more frequently than they are on planes. I.e. any given school shooting or plane ride IS more risky than any given bath or car journey, but the latter occur much more frequently.
Not sure exactly what the numbers are.
Cars & trucks 5.75 deaths per billion miles
Commercial airlines 0.06 deaths per billion miles
In my own personal case I put about 10,000 miles on my car each year, and I travel about 50,000 - 80,000 miles on an airplane. So airlines are still safer as I don't travel 2 orders of magnitude farther on them.
[1] http://money.cnn.com/2015/05/13/news/economy/train-plane-car...
[1]: https://www.statisticbrain.com/airplane-crash-statistics/
The same page gives the odds of dying in a single airplane flight as 1 in 29.4 million. The odds of dying per mile driven are approximately 1 in 174 million. On that basis, your 10 mile journey has a risk of death of approximately 1 in 17.4 million - death is 1.7x more likely.
There's probably a bunch of other ways to interpret the statistics - car miles aren't created equally either, there's a difference between long haul and short haul flight fatality statistics, there's a big difference based on where the airline operates. It's (like all things in life) not very clear cut.
And if you qualify the airline deaths based on journey type then you also need to qualify the car trip by journey type. As he is presumably sober, not tired, using a seat belt, not speeding 30+ MPH over speed limit etc his risks of dying over 10 miles is less than half that of the average 10 miles driven.
So, his risk of death per trip is actually higher when flying.
Just a quick Google search and the first results give me this info:
>Nearly 1.3 million people die in road crashes each year, on average 3,287 deaths a day. An additional 20-50 million are injured or disabled. More than half of all road traffic deaths occur among young adults ages 15-44.
From this page: http://asirt.org/initiatives/informing-road-users/road-safet...
Of course unmanned driving tests will happen at a very, very small fraction of the amount compared with manned drivers we have to pull stats from but I don't think it will take long, if the unmanned tests persist enough to conjure up the required amount of data needed to make a legitimate comparison to prove that unmanned driving will be markedly safer.
The only person who died in a road fatality in phoenix that day was killed by a self driving car.
Uber, with their recklessness, is inflaming them a lot.
I looked up what numbers I could find, according to the OECD's Road Safety Annual Report 2016, in 2014 the US averaged about one death per 150 million vehicle-kilometers (vkm) (6.7 killed per billion vkm).
According to Waymo's own page on the subject, they have driven "more than 5 million miles" (8mvkm) since 2009[0] and they seem to be by far the lead vkm producer in the autonomous space[1]. Just to be as safe as 2014 US human driving they need to clock in an other 140mvkm with no more than one fatality, and even then that could be debated to be in more selected conditions than human driving.
Don't get me wrong, I'm a big believer in autonomous driving and I think it would be a great, great thing, but saying it is already safer than human driving is at best misleading.
[0] https://waymo.com/ontheroad/
[1] https://www.theverge.com/2018/1/31/16956902/california-dmv-s...
Perhaps, it is easier to speak of things like "short term life lost vs long term life's saved", as long as it is other people losing lives.
You have to be quite shortsighted to say "no".
In addition, not providing any data there or alternative is just misleading.
Ex: I would NOT accept increasing the risk by 10 in the next 5 years to reduce it by 30% afterwards. I would NOT accept increasing the risk by 2 during 5 years if we could instead just increase it by 5% during 10 years and get the same result. I would NOT accept even having the same probability if we could first learn more and take 2 more years and a few billions to get a more stable technology before launch.
Yes, many times yes. The promise of self-driving cars is that they are totally safe, that is, nothing will happen except in exceptional, unforeseeable circumstances.
The reason for this is, in a self-driving car, nobody's in charge, and therefore, any incident (let alone, a fatal accident) is immensely more complicated than the same incident with a human driver.
(It's obvious that a self-driving car with a safety driver at the wheel doesn't provide any benefit whatsoever and isn't, in fact, a self-driving car).
A self-driving car needs to:
- transport people who are unable to drive or are "irresponsible" in the legal sense (kids, seniors, drunk people, injured/sick people going to the hospital, etc.)
- drive itself with absolutely nobody onboard
The analogy with trains is flawed because trains have drivers that deal with problems when there is one. Even "autonomous" metro trains are monitored individually.
In fact, save for special conditions like a closed network on a campus, etc., I'm less and less sure self-driving cars will ever be a thing.
Do they? There are other technologies that started out less safe but were accepted and became safer after acceptance, further boosting their popularity. Planes and cars would be examples and mills in the industrial revolution, but there are undoubtably others.
I'd say an automonous car which can completely fail to react to a person in its path is firmly in the "therac-level of failure" category - even if on average they crash much less than humans do. They should all be taken off road and only allowed once we determine through rigorous industrial-level proof testing that it can't happen again.
You also believe that it's possible to do lab tests so good that real-world testing will not be necessary.
Finally, you believe that the radiotherapy machines are 100% safe right now, and will never have a life-threatening problem again. Right?
2) You can do real-world testing without testing on public roads and endangering everyone around you - which is what Tesla was/is doing - all their cars were gathering data which wasn't actually used for autonomous driving yet, but they were able to see what the car would have done had it been running in autonomous mode.
3) No, I do not believe that. I do believe that a certain category of problems has been completely eliminated by rigorous testing, certification processes and engineering practices that prevent those problems in the first place. That's not the same as saying that there will never ever be an issue with a radiotherapy machine. To bring the topic back to self driving cars - the processes should be developed that will ensure that the car cannot not react(like it did in this case) when there is person in its path - it should be physically impossible from the software point of view. If the hardware cannot provide clear enough data to guarantee that, then it simply shouldn't be allowed on the road.
The driver assists you talk about will increase the safety of human driven cars to the point the self driving safety argument becomes less and less relevant.
The idea of a fully autonomous self driving car that can pick you up from your cabin by the lake and drive you into the city is in "flying car" territory for me. The AI required to execute that journey is so far off that the society could have changed enough to make the idea obsolete.
Self driving approved routes are the only way I see this working. Eg. goods haulage or reducing congestion in city centers.
Yes, they can, and, in fact, many operational machines do. You probably mean should not, but even that is clearly not the standard our society applies. Except for machines designed to kill people, we generally try to mitigate this to the extent reasonable, but lots of machines that have killed people are retained in operation in the same configuration.
I agree that they shouldn’t, but they certain do. Strangely we seem fairly ok with devices that kill us when we use them wrong (eg cars, power tools, weapons) but we are outraged when devices that make decisions for themselves kills us. For the victim it’s little different, but somehow it is.
When more cars are autonomous, we may also see news about fatal incidents fade into the background noise.
So yes, I am discounting the safety driver entirely. Autonomous cars need to drive themselves safely. That person was in the driver's seat as a scapegoat in reserve.
2) The train drivers in Europe are specifically hired to do just that. The ones in Netherlands are tested yearly. They're put on suspension when they run a red sign. Every so often they have to go through a special test for their attention span training. Uber obviously didn't do any of that.
The woman had crossed two lanes of the road before coming in front of the car - ie. fairly unrestricted range of view. A human driver paying attention would almost certainly have seen her and stopped or at least slowed down to a level where this would not have been a fatality.
No excuse for a modern car at only 38mph.
But frankly, I think grouping Uber together with more responsible companies does the whole industry a disservice.
I agree mostly with what you are saying, just not the undertone that this infortunate accident is somehow evidence of AVs "causing" more deaths than human drivers.
So yes, it's quite possible for autonomous vehicles to cause vastly more deaths than human drivers even while being slaves to the rules. Hell, you could do it too as a human driver if you removed your conscience. Just don't hit the brakes the next time you have the right of way and a jaywalker steps out in front of you. You'll easily kill someone within a week and have it be their "fault".
That's why I think the real self-driving car problem is a very special case of the Turing Test--one that might be more difficult to win or solve.
That's the thing - every driver has faced those situations, hundreds, thousands of times. Odds are very good a failure one of those times would have significant penalty for any driver, even if it wasn't their fault.
That doesn't mean that the Uber system wasn't dangerously defective and that a fit foservice self-driving system would have noticed in time to react (which may not have been sufficient to avert a collision, and may not have prevented a fatality, but given the speeds involved would have significantly reduced the risk of fatality.
That the pedestrian should not have entered the roadway where and when they did may relieve Uber of legal liability for this collision, it doesn't stop the failure to react at all from being evidence of a critical safety flaw.
And, had the same thing happened at an equally poorly lit, not specially marked, intersection in many jurisdictions (including California, not sure about Arizona), Uber would be legally at fault because of pedestrian right of way. If Uber's system can't see and react to people in dark clothes at night, it's not even remotely suitable for use, at least at night.
Interesting! What reasons do you have to think this is an unpopular opinion? To me, it sounds like an obvious conclusion.
Or they made a mistake or were distracted, like millions of people every day.
That argument might be convincing if the car had detected the pedestrian, slammed the brakes and evaded, but still could not overcome physics in time to prevent fatal injuries. But that's not what happened.
We could have built self-driving cars that mindlessly follow the road no matter what fifty years ago, but we didn't, because that kind of vehicle has no place on a public road. What changed now is not that the same danger is suddenly ok because it's 2018, what changed is that some self driving cars are supposedly much better now. Those that are not need to continue to be confined to the test track.
This isn't about autonomous cars as a general mode of transport, it's about Uber's autonomous cars specifically.
If Uber's cars are more dangerous than human-driven cars, then Uber's cars shouldn't be on public roads.
Waymo's cars can continue to operate since from all reports it appears they've been appropriately conservative when managing risk.
And don't blame the pedestrian. There was plenty of time to stop or swerve to keep this woman alive, but Uber's systems and "safety driver" failed miserably.
You have never been in a serious car wreck. I've been in three in my life and I am telling you, any human being that wasn't a racecar driver in a state of flow would not have been able to stop the car before hitting that pedestrian. It's terrible, yes, and Uber's autonomous vehicle should have stopped, but it simply wouldn't be possible for a typical human being to take appropriate action in that window of time when the pedestrian came out of the shadow. One way or the other, whatever "action" you think someone could take in that particular moment would have resulted in an injured or killed pedestrian and/or an injured or killed driver.
In this specific situation, Uber's autonomous vehicle was no more dangerous than a vehicle driven by a human being.
People do this all the time with things like this. Car wrecks, combat, disasters, fights, emergencies. Everybody has the inside track on what the failures were and why they wouldn't have done it that way and why they wouldn't have failed in that situation.
Furthermore, the driver did not have his eyes on the road, he was looking at a display, or probably his phone. I'm not exactly sure, but it doesn't even look like he had his hands on the wheel. Which isn't surprising, like, at all. You cannot sit somebody into a car without anything to do for hours, and expect them to be able to step in fast enough. If you want those people to stay alert, you have to take the appropriate steps like driving in pairs and extremely frequent rests.
If the safety driver had been paying attention she would have seen her and been able to stop.
If visibility really was that low then they shouldn’t have been driving that fast - drive to the conditions.
Automated transit is protected with platform screen doors, at least in Paris, France. This is a very good example of responsible automation. Also the EZ10 driverless bus which travels at 20 kmh and the Waymo driverless cars which also cruise at low speeds.
It's possible somebody said that they won't be held criminally liable. I would find it quite a stretch for anyone to say they won't be open to a pretty devastating civil action.
We know that level of safety provided by a driver is possible. But higher level is for now just speculative, even though additional sensors exist. You'd be punishing developing technology for not achieving the perfection yet.
Humans don't "see" stuff just because the light is entering their eyeballs. A human driver could have been checking their mirrors, the radio, distracted by another pedestrian on the other side of the road, and or just plane not registered the person. The motorcycle safety course teaches you to not use eye contact as a means of determining that a driver "sees" you. That's right, an attentive driver actively looking for on coming traffic during daylight can "see" right through you.
I'm really, literally sure she would not have been hit by that car had she not walked out in front of it..
You don’t get to hand wave away a death with the justification that the tech is better “on average”. The tech gets a pass when they demonstrate that they took reasonable steps to protect the public.
That means not shipping half baked product with crossed fingers so they can try and increase their stock price.
By that logic each time Uber driver kills someone you should be angry at Uber for cutting corners by hiring shitty drivers.
Fascinating how narratives and bias work.
The police reported that the pedestrian, wearing black, entered the roadway randomly and unexpectedly, directly in front of a moving vehicle, and concluded that a human driver would not have reacted in time either. The police reported the accident as "unavoidable", and showed that there was a crosswalk very close AND that there was signs advising pedestrians not to cross out of crosswalks right there too. "It's very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway"
If you had the proper context, you would have said "a human killed themselves" I think because no matter who or what is driving, when you jump in front of a moving car, you choose death.
People jump in front of trains all the time, and it sucks but it doesn't mean trains are bad or their conductors are murderers or anything.
Now having said that:
- I think their super-human system should be capable of reacting to what humans cannot, and should have reacted here
- I think Uber is not a good technology company and has zero chance of making a successful self-driving car themselves and all of this is a waste of money driven by the vanity of their former CEO who fancied himself Musk
Picture of where it happened: https://imgur.com/gallery/XQrAB
Also, the ISO is BLOWN OUT on your image to create the false perception of a well lit scene, you can tell by the significant pixel noise that incredible amounts of digital amplification have been applied to this image.
This is a better approximation of human sight: https://i.imgur.com/djOuoZr.jpg
The image you linked is just so digitally distorted that it's very misleading. Heck, your linked image is so distorted that the sky is literally grey!
There are lots and lots of false positives a self driving car has to disregard just to make its way along any street. Mailboxes on curbs, random parked cars, bridges, signs, and so on. I think the car unfortunately made a bad decision.
To say the LIDAR/RADAR or IR cameras didn't pick the person up is to say that these cars are woefully under-equipped to sense their environment.
I think the Police report was just working off of current operating Police knowledge and expectations of the law.
My overall point is simple, the car almost certainly saw the person - it just had to have, but the way it evaluated what to do about it was wrong because evaluating reality, as it turns out, is incredibly complex.
She did not enter the roadway directly in front of the Uber. She entered the road from the left and had to cross multiple lanes to get in front of the Uber, which was in the right lane. There were no obstructions between her and the Uber during all of this, and even if the road had not been well lit by streetlights, she would have been in range of the Uber's headlights for over 4 seconds before the collision (the low beams on that model car have a range of 250 ft, and the car was traveling 55 ft/second).
> The police reported the accident as "unavoidable", and showed that there was a crosswalk very close
The nearest crosswalk was 300 ft away, which I would not count as "very close".
That police report is at best sloppy and unwilling to do anything other than take limited evidence at face value and at worst an active attempt at spreading misinformation.
The pedestrian was crossing from the median across two lanes of traffic, and was struck on the right side of the right-hand lane of traffic (at least 75% of the way through crossing the road). Even the video in question (which, as many have noted, is far worse quality than anything a driver would be seeing) shows the pedestrian a good second before impact, which is enough time to at least initiate emergency braking even for a human--and the car shows no sign of having done this.
The police are biased because of the political situation in the state where the governor persistently courted Uber with relaxed rules.
> the pedestrian, wearing black, entered the roadway randomly and unexpectedly
People do random and unexpected things all the time, often while wearing dark clothes.
Claiming someone has a bias doesn't reflect well on your argument.
What safety system? I haven't read anything saying this, so any source would be appreciated!
However, my point is what happens within 3 meters is irrelevant, the car needs to look ahead and brake much sooner than the last 15 feet to avoid hitting stuff.
https://arstechnica.com/cars/2018/03/police-chief-said-uber-...
This is enough of a mitigation to appease the public and the regulators.
This Wikipedia entry covers many European countries. https://en.m.wikipedia.org/wiki/Jaywalking
Uber's model is based around selling at a loss to undercut Taxis, Lyft and other ride sharing companies, just enough to keep the investment capital coming in, with the hopes of ultimately cutting costs by removing the driver.
Even with Waymo, GM, Tesla and others, I don't think we're going to realistically see wide scale deployment for autonomous vehicles within the next ten years. They have money to burn. Uber doesn't.
There's a lot of tax money going into that research too and it really needs to be fed back into the public transport infrastructure that's crumbling around the US. I wish research on autonomous road vehicles stayed in the private sector. They're cool tech, but they'll be useless if systems like the NYC Metro and Chicago CTA fall apart.
I feel self driving cars are a stop-gap for better public transport inside smarter layouts of cities where cars are not the primary method of getting people and stuff around.
The only good fit for autonomous cars I see is long distance suburban commutes, or highway commutes, which is more a symptom of suburbia than a smart way to layout a city.
Part of my reasoning involves the dispelling the assumption that we have a right to a car and road based travel option, so I get that I am probably in a minority of thought process.
I also love my cars, love travelling on the open road and love combustion engines. But I would give it up for a safer, cleaner and more personable, community focused city.
We did that for cars, though, so it's not unprecedented.
I guess this is just a step in the evolution of per-individual transport, and eventually we will get rid of roads and automobiles as we know them for self-locomoting ambiguous transport pods that don't need big asphalt/concrete roads.
We waste so much space to roads and bespoke parking lots.
There's a great Wendover video one this:
https://www.youtube.com/watch?v=-cjfTG8DbwA
Transportation is the single greatest factor in reducing the number of people in poverty in a city. This isn't just opinion, there are a number of studies that suggest this. American can greatly reduce poverty, pollution and gridlock if we got over our apprehension to rails.
I think we might be in agreeance, and I am all for more rail and public transport. My little prediction was just musing about a possible outcome should we follow the autonomous car route, which I think would be the wrong choice.
I am happy to see that there are some projects working on alternatives (or I guess in some ways, bringing back what used to work).
With extensive rail comes easier cycling infrastructure as well, as it just follows the rail and is separated from the car traffic. It's a much more positive outcome for many reasons.
I see no reason (in this article, at least) to suspect that Uber won't re-apply at a later date when they have a better handle on what went wrong in Tempe.