Fatal crash in China: [1] Rammed into street sweeper at full highway speed.
Non-fatal crash in California.[2] Rammed into fire truck stopped on freeway.
Tesla's system does not detect big solid objects in front of the vehicle. It's "recognize rear of car", then "brake for car". No recognition, no braking. Machine learning at its finest.
I know, I've posted this before, but I keep seeing denial.
[1] https://www.youtube.com/watch?v=fc0yYJ8-Dyo
[2] https://abcnews.go.com/Technology/utah-driver-slammed-tesla-...
I'm aware that would be creating an exception for an exceptional situation and not really a root cause solution, but still.
That’s why other companies use lidar, which is much better at object detection, but has its own problems (like fog or heavy rain, for example)
The issue is mostly your last point. The radar sensors used have low resolution and would have trouble distinguishing a turning semi from an overhead sign.
I agree that when you can't handle this kind of situation in a tesla it really challenges the idea that we'll have total self driving in a year. There must be a lot of software that is under testing that could be more reliable than what they have today, yet the issue is still there. The last part (5 or 10%) of regular software dev is really hard, and it should be even harder for self-driving, never yet approved or safe enough self driving software.
In case of the first trailer crash it apparently looked similar to an overhead sign to the sensors, also approaching a stopped object would be AEB territory, not ACC, which doesn't work well at higher speeds.
Tesla later pushed a software fix for the AEB to trigger even if only the radar alerted.
https://electrek.co/2016/07/01/understanding-fatal-tesla-acc...
Step 2: if stopped object is immediately not in front of the car, otherwise, ANALYZE IMMEDIATELY because a stopped object is right in front of the car.
It's not rocket science, and it's something so ridiculously basic that Tesla's failure to do something this simple borders of gross criminal negligence. It's something their competitors already do; in fact Waymo's problem is that it's too sensitive to stopped objects in front of the vehicle, hence the Waymo's constantly stopping and starting or driving very slowly...but that's the better and safer way to do it, especially when you're testing on real streets with third-party vehicles.
The car always has still-standing objects in front of the car (the road, anything around the road, barriers, anything on opposite sides of barriers, things in the direction of the ar but the road is turning etc.
The car has to know that the road doesn't slope up over the truck or around it. I'm guessing that's where the split-barrier fails occur.
It really needs to create a real-time 3d map of the entire environment (including what's visible from eye-height, not only front of hood) and map a route through that.
Right, none of which are directly in front of the car when the objects and care are plotted in 3d space. Easy (but not trivial) to do with LIDAR but not yet doable with purely visual systems like "Autopilot." Which is my point, and which is the reason every other self-driving car company uses a combination of sensors to get data about car's surroundings.
Once they are sophisticated enough to have an actual world model (something which, to my knowledge, only Waymo actually has ATM) they can stop ignoring static objects. Until then, they would keep slamming the brakes every 30 seconds for all the stuff they don't understand.
Of course this seems like a really hard thing. Probably with enough layers and with the right learning tasks it might be able to do it eventually. But the probability for noise and various learning artifacts is enormous, when you want to learn a sort of model of everything road related.
Then it should be properly marketed as autocrasher, not autopilot.
An autopilot that requires that you have your hands on wheels and pay attention at all times is missing the auto part of the autopilot (reminder that auto means self). So such a system can't be called an autopilot.
How many times did the driver have to intervene to avoid the crash? Tesla provides no data about this. But next time you hear “AP is so safe it only crashed 3 times” just remember that student drivers also almost never crash. As long as it needs constant attention, not considering this in the statistics makes them irrelevant.
It seems this is a classic estimation problem, and we need performance data points to fit a curve. Do we have anyting? Do we even know if this is a problem where we have a chance at accelerated compounding (so exponential improvement rate)?
I think the self-driving car efforts are dramatically underestimating the effort required to close those gaps to zero at least in the hype machine but also perhaps in the daily working cycle. (That's all well and good; I'm sure that any audacious engineering program should discount some of the hardest parts lest everyone get discouraged and never begin.)
Well, there are just a lot of them, outside the lane markers, and those never kill you. And while the systems surely try to classify parked cars as well, detection quality isn't under much scrutiny. Then when that one exception is sitting in the lane, good luck, your life depends on a second class feature.
Anecdotal evidence suggests that a similar perception vulnerability exists in long distance cyclists who spend many hours on the own road: lanekeeping and pothole/junk avoidance get more and more delegated to subconscious processes which work with eerie (and enticing) reliability. But that "mental autopilot", tuned to the open road, knows the back of cars only as something that pulls away from you, the simple model knows them as something not worth "raising an interrupt" over. Similar learned routines certainly exists with highway driving as well, but when driving, there isn't a class of objects which is routinely there but almost universally not a concern, like backs of cars are for a cyclist. Reports of that cyclist hitting a car parked in the open road always earn a lot of ridicule, but it's not quite as surprising as it seems at first glance (the very majority certainly succeeds at "disengaging" in time, but the exception make the news, same thing in every field)
I also wonder if this statistic includes situations where the autopilot had disengaged immediately beforehand and the human had failed to respond.
What’s the human disengagement rate?
As it happens, my uncle drove for a living and I recall he had a plaque for a million miles without an accident.
And let me pre-empt the replies by saying yes, what matters in safety is the statistics, not a single data point. And let me pre-empt the replies to the reples by saying yes, but the statistics on self-driving cars are not at all encouraging.
I suspect Autopilot's running-into-semis rate is worse than humans', for three reasons:
(1) Tesla has been cagey about their Autopilot safety stats. If Autopilot was clearly safer, I think they'd be less cagey. Absence of evidence is evidence of absence. This is just speculation, of course.
What concrete data they have shared is summarized here: https://www.tesla.com/VehicleSafetyReport
Teslas with Autopilot on crash roughly once every 3 million miles. To their credit, that's really good! However, they don't say: (1) how often Teslas crash on freeways with Autopilot off, or (2) how often drivers have to take control to prevent Autopilot from crashing.
(2) The fatal crash in 2016 was also with a semi trailer. 2/4+ fatal crashes having this failure mode suggests this failure mode is especially high for their vision based system. Not a good look.
(3) Pretty much all self-driving companies have disengagement rates way worse than humans. Waymo is leading the pack - by far - at something like once every 10,000 miles in California. There are tons of caveats in interpreting these numbers, and they should probably be treated as an upper bound, since it's at the discretion of the company to only self-report disengagements that would have led to crashes, I believe. But human crash rates are something like once every 300,000 miles. So even lidar based systems are an order magnitude away from humans, by one super rough measure.
Lastly, a small correction to the title: This is the third known US autopilot crash. I believe there was a suspected autopilot crash in China some years back as well, and I'm not sure about other countries.