Generally speaking, the average human driver causes a “ding” every 100k miles, a insurance worthy event every 250k miles, a police worthy event every 500k miles, and a fatality every 60M miles. Assuming an average driving speed of 60 mph, which is obviously an overestimation, those are 1.6k, 4k, 8k, and 1M hours respectively.
Evaluating a L2 autonomy system is somewhat challenging since in truth you are evaluating a human+autonomous system, so you should expect the overall safety to increase unless the autonomy system is actively making things more dangerous. However, for a L4/L5 autonomy system, which is Tesla’s goal, we can reasonably approximate the quality by miles per intervention as an intervention is a failure of the autonomy.
As a reasonable baseline based on video evidence of FSD, we can assume that each intervention prevents at least a “ding”. Therefore, we can evaluate the quality of FSD relative to a minimal L4 system as the ratio of time per disengagement versus 1.6k hours. Using those same videos as evidence, we see the FSD system requiring intervention every few minutes, i.e. <10 minutes, on average. That is 1/10,000 of the minimal quality expected from a fully functional L4 system.
So, for Tesla to create a minimal acceptable L4 autonomy solution that would actually usher in a age of safe self driving systems, they need to improve the core functionality of their beta product by 10,000x. It is that gap between what they are delivering and what they still need to achieve that is outrageous.
In the mean time, we could start to get a better handle on the safety profile of these systems by better classifying the interventions.
Something like these:
* Intervention due to unreadable conditions (e.g. unpainted lines, a new area with no training data, failure of LIDAR / camera, w/e)
* Human intervention due to not feeling safe
* Intervention due to unsafe condition (e.g. imminent collision, or just hit something)
* Human intervention for other reason (e.g. human is bored, or human changing destination, or human wanting to take another route)
---
#1 could certainly lead to #2 and #3, but is more of an indicator that the car is in novel conditions not trained for, or has hardware failure. #1 is the low hanging fruit (relatively speaking), and #2 is where the majority of the work lies.
#3 is where the system has actually failed.
And #4 need to just be discarded.
> we can assume that each intervention prevents at least a “ding”.
I don't think we can be sure that the vehicle wouldn't have corrected itself in time, nor how many would have been much worse than a "ding". This is where some closed course testing with simulated pedestrians and bicycles etc may give more evidence. In evaluating self driving software, knowing what the consequences of a failure are is important. As you point out, safety drivers prevent this assessment.
> based on video evidence of FSD
I think that there is selection bias in the readily available footage. Firstly, the short clips are likely to be where there is an incident. The longer ones are usually beta testers deliberately choosing tricky routes to test the FSD. I don't see anyone posting videos of a 3 hour freeway drive where nothing happens.
But my point is just that this is complex, and we should be much more careful than we are being.
It's not like Tesla has to release new features every year, they just feel they need to to continue to pump up and justify their stock price.
EDIT:
I misread the parent comment, I’m not opposed to just banning self driving cars as self driving cars likely won't make everyday folks' life better.
Just because this instance was regarding a bicyclist doesn't mean it's only a danger to bicyclists, there are still ways for people to die if the car were only interacting with other Teslas.
I'm more of the mind that cars should become the second-class citizens they ought to be in cities and that I should be able to bike around the city without constant threat of death, that dream remains far off in american cities.
Why is that fire codes and aviation regulations are written in blood, but traffic violence is treated as a normal, necessary cost of modern life? When a biker is killed by a driver who wasn't signaling and cut them off on a right turn (as happens to me on a daily basis in SF), that should trigger immediate redesigns of intersections and streets across the city, for that was an entirely preventable death.
But nope. Go ahead and test your fundamentally flawed, unproven death machine on public roads.
I gotta say, I am really not that keen on being killed by a white tesla.
What rules did Tesla break?
Also, remember that time Waymo killed a pedestrian?
waymo never killed a pedestrian
Yeah, it was Uber, my mistake.
But that's an example another self-driving company that killed people. It could have easily been any of them -- they have all had accidents that could have resulted in death, and just got lucky. Tesla is not special here.
Previously: https://news.ycombinator.com/item?id=30182141
It's to be expected that people will occasionally die in car accidents. Some things are hard to avoid. If someone on a bicycle comes shooting out into a busy street when they don't have the right away and no one could see them coming, they're probably going to get run over. My general expectation for self-driving vehicles is: don't create unsafe situations that weren't there before, don't behave in unpredictable ways, and if you can't avoid running someone over at least do it in a way that the consequences fall on the person doing risky things. In the above example, swerving to miss the biker is fine if it can be done safely, but swerving to miss the biker and instead running over someone on the sidewalk instead is not. (Or, in other words: if you can't avoid an accident, at least be predictable and try to obey the general principles of right-of-way.)
Swerving at a bike for no reason as in the video above should result in suspension of self-driving features (assuming the video is real and wasn't staged in some way).
I can see this being a reason to ban a system from operating at SAE level 3 or above, but isn't the entire difference between level 2 and level 3 that things like that are expected to happen at level 2 and below, hence the reason for drivers having to always be ready to immediately intervene at those levels?
However, self driving is definitely guaranteed to fail if it can't be tested on public roads. And that's really the challenge here -- it can only succeed if society is willing to take the risk.
Sensible legislation is needed to balance innovation and safety for self driving cars on public roads, but not zero. We don’t even accept zero pedestrian hit and run deaths yet…
California, Texas, and Michigan legislators have already bought in to the benefits that these companies will bring.
That's equivalent to a ban.
>If entities want to innovate, let them innovate within safety parameters.
Sure, but the safety parameters have already been established[0] at 40,000 deaths a year, or less, for transportation equivalently convenient and functional as compared to a manually-driven car.
[0]By the crowd-sourced wisdom of democracy.
Are you saying that, despite transit alternatives that do not lead to excess deaths, that deaths due to self-driving cars is worth the cost? We have existing solutions that don't result in these excess deaths, but because of some reason, we don't want those solutions _and_ we want to kill more people just to birth this technology? _Why_?
> [0]By the crowd-sourced wisdom of democracy.
... Which bill are you talking about? I don't remember my Rep or Senator voting on allowing me to be a casualty of the self-driving industry. The fact that automobiles in the US result in 40k deaths is itself a travesty. Even Canada is doing better than the US here. Pointing to the worst driving safety record in the G8 and one of the worst in the G20 isn't exactly proof positive that the American automobile system is safe, it's just proof that we have grandfathered an extremely unsafe system here.
The existing solutions <As chosen by the collective will of the American people> cause 40,000 dead Americans every year. Other 'Solutions' are not. Don't ask me why they are not, ask your neighbor, and your congressman -- but they are not. Also ask yourself how many collective minutes spent waiting for a bus is a traffic fatality worth? This isn't rhetorical, the answer is a number.
>Which bill are you talking about?
Lets start with the United States Department of Transportation, followed shortly thereafter by your state's department of transportation.
>it's just proof that we have grandfathered an extremely unsafe system here.
All the reason why a potentially new system should have to meet the safety standard of 'less unsafe' and not 'perfect'. Literally every day the adoption of self driving is delayed kills 109.6 people[0] because that's one more day of the status quo. To save lives we should be damn near throwing people in front of Teslas to help them train instead of wringing our hands over the matter.
[0]As much as, divided by the relative eventually safety of a fully developed self driving. As Americans, more globally of course.
Americans choose automobiles because of zoning laws mandating massive setbacks and minimum parking requirements. There's nothing in the DoT or my state's DoT that mandates use of a car. It's all due to American city planning, which is driven by auto friendly city planning policy. So again, why is transit out of the question here?
Self-driving cars aren't an incremental improvement away. We have no idea how many more deaths are needed to mature the technology. So why are we bending over backwards for it when there are proven solutions to move people without casualty? If self driving can innovate without casualty, then be my guest.
You would kill tens of thousands of people a year with that policy decision.
The unstated assumption in this is that self-driving is feasible with current technology and without enough training will be better than human drivers.
That's just not realistic, not even in Europe.
Why wouldn't it be realistic?
If a state government or the US Federal government tried to ban cars, they would be out of power very quickly, even in the most liberal parts of the country.
All of the companies that are testing self-driving cars have had accidents with other cars and pedestrians. Some of them resulted in death. Others did not, but easily could have, and eventually their luck will run out and they will also cause deaths.
Who would invest millions of dollars into developing a technology when a single mistake could flush all that money down the drain?
Furthermore, no self-driving system will be perfect even in a mature stage of development. People will still die, and they may die because of mistakes a human driver would not have made.
If society isn't willing to accept that sacrifice, then let's just ban self-driving cars.
But let's not try to have it both ways -- let's not allow self-driving while also holding the view that risk of death is unacceptable.