Whoever owns the algorithm. Or at-least in whoever's name the license/permission was issued. If its an organization, the top management signing off on this has to take the blame.
>> It's physically impossible for a human to intervene on the timescales involved in motor accidents.
That remains true even without any automatic driving tech - you are responsible even for accidents which happen too quickly for anyone to intervene. Obviously if you have some evidence(dashcam) showing that you couldn't avoid the accident you should be found not guilty, but the person going to court will be you - not the maker of your car's cruise control/radar system/whatever.
The assists are certainly helping more than anything, so I feel that the Mazda is much safer to drive in heavy traffic than the older Outback.
The cruise has autonomy over controlling the speed only, and applying brakes, but it is still autonomy. Of course since my hands never leave the wheel it may not fit with what you have in mind.
Having said that, Mazda (or Bosch?) really nailed their radar, having never failed to pick up motorbike riders even though the manual warns us to not expect it to work.
I feel more confident in a system where the ambition is smaller, yet execution more solid.
Fwiw I also tested the AEB against cardboard boxes driving through them at 30km/h not moving accelerator at all, and came away very impressed by the system. It intervened so last second I felt for sure it wasn't going to work, but it did - first time was a very slight impact, next two were complete stops with small margins.
This stuff is guaranteed to save lives and prevent costly crashes (I generally refuse to use the word "accident") on a grander scale.
I do love that even though they have a ton of driver alerting features, hands have to be on the wheel at all times.
Either you have full autonomy without hands or you don’t. There is no middle ground, it’s a recipe for a disaster.
There are several states is USA that are more progressive than others (CA namely). But with many working groups in and around the legal side - it hopefully will be a thing of the past.
In Australia, they are mandating by some year soon (don't have it on hand) that to achieve a Safety Rating of 5 star, some level of automation needs to exist. Such as lane departure or ABS will become as standard as aircon.
Let me put it this way - my Mercedes has an emergency stop feature when it detects pedestrians in front of the car. If I'm on cruise control and the car hits someone, could I possibly blame it on Mercedes? Of course not. I'm still the driver behind the wheel and those systems are meant to help - not replace my attention.
What we have now in these semi-autonomous vehicles is nothing more than a glorified cruise control - and I don't think the law treats it any differently(at least yet.).
Now, if Uber(or anyone else) builds cars with no driver at all - sure, we can start talking about shifting the responsibility to the corporation. But for now, the driver is behind the wheel for a reason.
The San Francisco Chronicle late Monday reported that Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway." (bit.ly/2IADRUF)
Moir told the Chronicle, “I suspect preliminarily it appears that the Uber would likely not be at fault in this accident,” but she did not rule out that charges could be filed against the operator in the Uber vehicle, the paper reported.
On top of the speedometer it has the GPS speed to compare as well, I can't see how there is any excuse for being over the limit.
The quoted stats from UK advertising were that at 40mph 80% of pedestrians will die from the crash, at 30mph 20% will die.
Had the car been doing just under the limt e.g. 33mph then there's a much better chance that the woman would have survived.
http://www.dailymail.co.uk/news/article-5332443/Police-chief... [30 January 2018]
https://www.telegraph.co.uk/news/2018/01/31/motorists-should...
Can you explain why you think this rule changed a decade ago?
Because sometimes that 10% is argued as a margin of error for humans supposedly not paying attention how fast they're going, but if that's the case then there's really no reason why the robot shouldn't drive strictly under the speed limit.
If you explicitly programmed a fleet of robots to deliberately break the law, then I think it's not enough consequences if you just fine for the first robot that gets caught breaking that law, while the programmers adjust the code of the fleet to not get caught again.
Consequences should be more severe if there's a whole fleet of robots programmed to break the law, even if the law catches the first robot right away and the rest of the fleet is paused immediately.
> Chief of Police Sylvia Moir told the San Francisco Chronicle on Monday that video footage taken from cameras equipped to the autonomous Volvo SUV potentially shift the blame to the victim herself, 49-year-old Elaine Herzberg, rather than the vehicle.
> “It’s very clear it would have been difficult to avoid this collision in any kind of mode [autonomous or human-driven] based on how she came from the shadows right into the roadway,” Moir told the paper, adding that the incident occurred roughly 100 yards from a crosswalk. “It is dangerous to cross roadways in the evening hour when well-illuminated managed crosswalks are available,” she said.
0) http://fortune.com/2018/03/19/uber-self-driving-car-crash/
Based on the layout of the presumed crash site (namely, the median had a paved section that would effectively make this an unmarked crosswalk), and based on the fact that the damage being all on the passenger's side (which is to say, the pedestrian would have had to have crossed most of the lane before being struck), I would expect that there is a rather lot that could have been done on the driver's side (whether human or autonomous) to avoid the crash.
> Herzberg is said to have abruptly walked from a center median into a lane with traffic
So that explains that. However, contrary to the thrust of your argument, the experience of the sober driver, who was ready to intervene if needed, is hard to dismiss:
> “The driver said it was like a flash, the person walked out in front of them,” Moir said. “His first alert to the collision was the sound of the collision.”
And also:
> “It’s very clear it would have been difficult to avoid this collision in any kind of mode [autonomous or human-driven] based on how she came from the shadows right into the roadway,” Moir told the paper, adding that the incident occurred roughly 100 yards from a crosswalk. “It is dangerous to cross roadways in the evening hour when well-illuminated managed crosswalks are available,” she said.
No human could brake that well, and simply jamming the brakes would engage the ABS leading to a longer stopping distance. Not to mention the human reaction time of 0.5 - 0.75 seconds would have prevented most people from even lifting the foot off the accelerator pedal before the collision, even if they were perfectly focused on driving.
I was taught that the entire point of ABS is so that you can just jam the brake and have the shortest stopping time instead of modulating it yourself to avoid skidding. Do you have any source to the contrary?
Older dumb ABS systems would simply "pump the pedal" for the driver, and would increase stopping distance in almost all conditions, especially single-channel systems. Newer systems determine breaking performance via the conventional ABS sensors and additionally accelerometers. These systems will back off N G's, then increase the G's bisecting the known-locked and known-unlocked condition, trying to find the optimum. These systems _will_ stop the car in the minimum distance possible, but very few cars use it.
Wikipedia backs me up but adds that it also decreases stopping distance on dry and slippery surfaces, while significantly increasing stopping distances in snow and gravel. I’m from a country with a lot of snow so that makes sense.
In terms of split second reactions, it's pretty much optimal still to just jam the brakes if you have ABS. It's much better than braking too little, which is what most non-ABS drivers would have done.
That said, the point of ABS is in the rare event that you have to brake full power, the system automatically help you do it at a near optimal (slightly skidding) without additional input, and you remain full steering ability.
If you don't have ABS you'd need to train that emergency stop ability on a daily basis to even come close.
Many cars, such as my POS Ford Focus, use a single-channel ABS system. These systems will oscillate all four brakes even if only one is locked. Combined with the rear-wheel drum brakes, the ABS considerably increased stopping distances on dry road.
Maybe it's society's fault, for building open-access roadways where vehicles exceed a few km/h.
As others have said in the comments, whole point of having technology is defeated if it performs worse than humans. Assuming vehicles were parked, a sane human driver will evaluate the possibility of someone coming out from between them suddenly and will not drive @40 Miles an hour speed.
If that's the case most drivers on the road are very far from "sane drivers." I've been illegally passed, on narrow residential streets, many times, because I was going a speed that took into account the fact someone may jump out between parked cars.
http://fortune.com/2018/03/19/uber-self-driving-car-crash/
> she came from the shadows right into the roadway
also we were told radars would have have solved exactly this limitation of humans
> Uber car was driving at 38 mph in a 35 mph zone
also we were told these car would be inherently safer because they would always respect limits and signage
> she is said to have abruptly walked from a center median into a lane with traffic
I don't know other driver, but when someone is on the median or close to the road I usually slow down in principle because it doesn't match the usual expectations of a typical 'safe' situation.
I've been advocating against public testing for a long time, because it's just treating people safety as an externality. Uber is cutting corners, not all company are that sloppy, but this is, overall, unacceptable.
This means that there's a fixed distance from which the optimal driver can stop a car doing xmph. Yes, an autonomous vehicle has a faster reaction time* to begin the stop, but no matter the reaction time, a stop cannot be instantaneous from any substantial amount of speed.
If it takes 20 feet to stop a car doing 20MPH, it will take 80 feet to stop a car doing 40mph. If there's a human between the initial brake point and 80 feet from it, that human will be hit, no matter who or what the driver is.
Or maybe it couldn’t, but then the whole « narrative « of the experiment is in serious jeopardy.
Not really. Self driving cars are supposed to be better than average human driver. That does not imply that they NEVER make mistakes.
I do not know specifics of this case, but a general comment: If somebody is hiding behind a bush, and (deliberately or by mistake) run in front of the car, there is no way the car can anticipate that. There is no way to avoid accidents in 100% of the cases.
I can think of many situations where I have avoided hitting pedestrians because of my avareness of the situation. Eg: Pedestrian with earphones looking at phone crossing against red light just because the left-turning wehicle in the left lane stopped for a red arrow while I had green going straight. Pedestrian mostly behind the car, just seen thru the window of the car.
Pedestrian behind high snow-walls going towards normal pedestrian crossing, no lights. Almost completely covered by the high snow-walls and a buss parked at a buss station 50 m away from the crossing. 50 km/h road. Since I had seen the pedestrian far away already I knew someone would show up there at the time I arrived there. On the other hand I would never pass a buss like that in high speed, pedestrians like to just run across in front of the buss. And high snow-walls next to a crossing is a big red flag too.
I live in Sweden though, where pedestrians are supposed to be first class citizens that has no armor.
This looks like the exact situation the selfdriving cars are supposed to be able to avoid. A big object in the middle of the street. I expect the car to try to avoid this even though the bike didn't seem to have any reflexes. If the LiDAR doesn't catch this, I don't think they should be out in traffic at all.
Yes, but this is a 4-lane roadway. I can totally imagine driving cautiously and slowing down near residential areas where houses are close to the road. However, this seems like a different case.
It, or the driver, could do more than just stop though. You can change directions, even at 38mph.
Then we have to get into other questions, would I as a driver willingly sideswipe a car next to me to avoid hitting a pedestrian? Is it reasonable to expect an AI to make the same value decision?
An algorithm however is a different deal, what should happen is already decided in an algorithm, so in some way its already settled who gets killed.
You can't avoid what is hiding.
0) https://www.nytimes.com/2018/03/19/technology/uber-driverles...
That would increase the chance a particular vehicle could avoid an accident which it couldn't, on its own, anticipate.
It would be much safer if all roadways with speed limits over a few km/h were fenced, with tunnels or bridges for pedestrian crossing. Arguably, we would have that now, but for legislative efforts by vehicle manufacturers many decades ago. Maybe we'll get there with The Boring Company.
"Most people" (which is, in reality, "most of my geek friends with high disposable income") shifts to "everyone" by the end of sentence. Also, my devices seem to know where I am...within a few city blocks: I do not like your requirement of always-on mandatory tracking, both from privacy and battery life POVs.
Even worse, this has major false negatives: it's not a probation tracker device - if I leave it at home, am I now fair game for AVs? And even if I have it with me and fine position is requested, rarely do I get beyond "close to a corner of X an Y Street," usually the precision tops out at tens of ft: worse than useless for real-time traffic detection.
Moreover, your proposal for car-only roadways is only reasonable for high-speed, separated ways; I sure hope you are not proposing fencing off streets (as would be the case here: 50 km/h > "a few km/h", this is the usual city speed limit).
Some do argue that speed limits for unfenced roadways within cities ought to be 30 km/h or less. And although fatalities are uncommon at 30 km/h, severe injuries aren't. I live in a community where the speed limit is half that. But there's no easy solution, given how prevalent motor vehicles have become. Except perhaps self-driving ones.
0) https://www.gps.gov/systems/gps/performance/accuracy/#how-ac...
Edit: spelling
[1]http://www.phoenixnewtimes.com/news/cops-uber-self-driving-c...
Accidents can still occur even if the car is perfect.
Not saying this is the case here, but it could be. As others have said, we need to wait until we know more before jumping to conclusions.
Thinking about it seriously, it may be shouldn't. Also these things could lead to a crash pile up with other vehicles coming in from behind.
Also my problem with this is that a human death is functionally treated as finding edge cases that are missing a unit test, and progressing the testing rate of the code... and that really bothers me somehow. We need to avoid treating deaths as progress in the pursuit of better things
Au contraire. Go read building codes some time. There's a saying that they're "written in blood" - every bit, no matter how obvious or arbitrary seeming, was earned through some real-world failure.
The death itself isn't progress, of course. But we owe it to the person to who died to learn from what happened.
http://www2.ensc.sfu.ca/undergrad/euss/enscquire/Vol10No2/pa...
Or, you know, don't jump on comments for their not explicitly addressing the hobby horse you're riding. Frankly I just wanted to express a better engineering context around the loss of life without getting into the political bullshit for once.
This implies they're using the same systems and underlying models. If one model hit a pedestrian because of a weakness in training data plus a sub-optimal model hyperparameter, and therefore was classified a pedestrian in that specific pose as some trash on the street, how do you share that conclusion with other companies models?
I don't think all self-driving cars have the same sensors.
If you have LIDAR model X, and they were using LIDAR model Y, will your system "magically figure it out?
If your car has cameras at 5ft high, and the data is from a camera 6ft high?
Sure, someone could release the data, but will it screw up your models more than it fixes them?
(I totally agree the data should be released, I'm just not sure other self-driving cars will directly benefit. Certainly they can indirectly benefit from it.)
The idea you present is possible, but I have to wonder how viable it makes the idea of self driving cars.
It may, it also might not. If sharing data fails, further methods would be needed - but it's a good start for figuring out what data should be recorded for comparison. If the data is entirely incompatible, then we should have regulation to require companies to at minimum transcribe the data into a consumable format after an event such as this.
> ... if the models interpreted the pedestrian as trash on the street, then how it responded (driving over it) is not inappropriate.
If the models saw anything at all, they should not have driven over it. Even human drivers are taught that driving over road debris is dangerous. At minimum it puts the car/occupants at risk - in extreme cases, the driver may not recognize what it is they are driving over.
If this isn't a case where the car was physically unable to stop - it's more likely the telemetry didn't identify the person as an obstacle to avoid at all.
The great thing about computer generated test cases is you don't need real footage of every hypothetical awful thing that could happen i.e. a truck losing control and rolling over sideways. These could be a stage 1 test -- a pre-requisite to real testing. Like the hazards test before you are allowed to sit the drive test with a real drive test employee [1] to make sure you're not a retard.
[1]: https://www.vicroads.vic.gov.au/~/media/images/licences/hpt_...
That being said, when the first real cars were introduced to the world and later improved upon there were many more fatalities than we're likely to experience with self-driving technology.
Every technology is dangerous. Every technology costs lives to some extent when spread across billions of people. I'm sure forks take more lives each year than self driving cars.
Weigh this against the potential lives saved. I posit you'd be killing more people with drunk drivers by slowing innovation than you'd be saving via luddism.
What's more, a lot of people seem to think Waymo's tech is further along. So not only does this project have to succeed, it is also the underdog. So no wonder they're aggressive.
We all make similar decisions all of our lives, but nearly always at some remove. (With rare exceptions in fields like medicine and military operations.) But autonomous vehicles are a very visceral and direct implementation. The difference betweeen the trolley problem and in autonomous vehicles is in time delay and the amount of effort and skill required in execution.
Plus, we're taking what is pretty clearly a moral decision and putting it into a system that doesn't do moral decision-making.
There's also NHTSA (which is indeed part of the DoT).
It looks to me like we don't need any new agency at all, just a very small expansion of NHTSA's mandate to specifically address the "self-driving" part of self-driving cars.
Then by all means lets stay at home and avoid putting humans in rockets ever again because if you think space exploration will be done without deaths you are in for a surprise.
The real news isn't "Self-Driving Car Kills Pedestrian", the real news is "Self-Driving Car Fatalities are Rare Enough to be World News". I'm one-third a class M planet away from the incident and reading about it.
They are rare only because self-driving cars are; I don't think the total driven miles of all self-driving cars are enough that even 1 fatality would be expected if they were human driven; certainly Uber is orders of magnitude below that point taken alone.
There are lots of fatalities from human-driven cars, sure, but that's over a truly stupendous number of miles driven.
There's different estimates from different sources using slightly different methodology, but they are all in the neighborhood of road fatalities of 1 per 100 million miles traveled. [0]
Waymo claims to have reached 5 million miles in February [1], Uber (from other posts in this thread) is around 1 million miles; the whole self-driving industry is nowhere near 100 million, and has one fatality. So it's way worse, as of today, than human driving for fatalities.
Of course, it's also way too little data (on the self-driving side) to treat as meaningful in terms of prediciting the current hazard rather than simply measuring the outcomes to date.
[0] see, e.g., https://en.m.wikipedia.org/wiki/Transportation_safety_in_the...
lol
Does the NTSB have regulation on how black boxes are allowed to function?
Would be better if the code and data was open for public review.
Making it public means more eyes on the data, which can lead to a better understanding of what went wrong.
This comment by Animats in the context of self-driving trucks is quite telling. He warns precisely of this danger.
Take a look at the cost of airplanes, even small ones.
So it's not just "don't drink and drive", knowing that they'll probably reoffend soon anyway. Every crash and especially fatality can be thoroughly investigated and should be prevented from ever happening again.
Hopefully there's enough data in the investigation so that Tesla / Waymo and all other car companies can include the circumstances of the failure in their tests.