Oh wow, it can stay in a lane and match speed! So can every car manufacturer. They’ve got a level 2+ system and cars won’t be safer until level 4.
No one in the public has enough data to say one way or another whether that is true and by presenting it as fact you are falling into the same trap that OP is warning about.
Agreed.
> Oh wow, it can stay in a lane and match speed! So can every car manufacturer. They’ve got a level 2+ system and cars won’t be safer until level 4.
I don't find this to be true at all. ProPilot is terrible compared to AutoPilot and many of the other manufacturers have the same limitations (or worse).
I second this - recently rented a fully-loaded Nissan Rouge and drove for around 700 highway miles - the lane centering feature was just awful compared to my Model 3's. It would often times give up on even slight curves, or fight me when I would try to regain control after it misjudged a curve. After a while I gave up on it and just used the automatic cruise control, which worked fairly well.
This is frightening. I strictly limit where I will let AutoPilot operate because it is not at all relaxing unless you're on the open road with light traffic. If this is the state of the art, FSD is decades away.
The fact that a majority of the public think it's the opposite way around, that skeptical opinions about self-driving cars must provide evidence, is the industry's succes- the triumph of marketing over reason.
Of course the industry has no such data. Research has shown that this data would take many decades to obtain:
https://www.rand.org/pubs/research_reports/RR1478.html
Personally the only justification I've heard for self-driving cars being safer is "computers are faster than people". Yes- but at what?
There is plenty of opportunity for computers to be better than humans. It is a hard problem though.
Faster at making the right decision - and the wrong decision too.
If you want to murder people on the roadway, there are lots of ways to do this that are easier than fancy adversarial attacks (which only fool one sensor type, at best... and are then patched).
With AutoPilot, Without AutoPilot but With Active Safety Measures, Without AutoPilot or Active Safety Measures, and NHTSA Average Vehicle
19Q3 - 4.34 2.70 1.82 0.50
19Q2 - 3.27 2.19 1.41 0.50
19Q1 - 2.87 1.76 1.26 0.44
18Q4 - 2.91 1.58 1.25 0.44
18Q3 - 3.34 1.92 2.02 0.49Driving assistants would improve safety certainly, but removing the driver and letting the assistant drive will need real world like tests, for now maybe you could have an independent group monitoring a number of Teslas and checking each time a human intervened and give a score on how bad the AI was.
Level 4 for example could be a blood bath, where cars drive all the easy situations, and hand over emergency control to out of practice drivers barely paying attention at a moments notice.
I believe our driving skills were better when we had manual transmission, and were more involved in overall operation. Hard to determine if safer, b/c of advances in crumple zones, airbags, etc as manual declined.
Level 4 requires that the vehicle safely stop in situations it can't handle.
The extremely sad part is that we could reduce the number of deaths until true AI arrives using simple tech that could detect if the driver is drunk or tired, and controversially forcing the speed limits(the car knows that this road is max 90Km/h and that the driver has less then 1 year experience so it will not let you get over 70 (local laws apply) unless you activate emergency mode - this would trigger some bad consequences if it was no emergency)
However I do pay attention and keep my hands on the wheel, I think autopilot + human is much safer than either one, but since the latest updates I rarely have to take control if not going in roundabouts or junctions.
I also had one instance a few weeks ago where I tried to enter another lane manually and the car took control from me and put me back in my lane as there was someone in my blind spot without their headlights on so I didn't see it.
I had two crashes in my lifetime. Both occurred when driving backwards out of a parking lot. No fatalities, limited damage. Good luck finding such situations while driving on a highway...
(that's data from the highest crash rate country where you were able see so many crashes that you develop a feel)
I’m ok if we assume that things are unsafe before we have evidence they are safe, but that’s not the same as declaring they are unsafe without evidence.
In such cases, demanding statistical confirmation that it is a significant problem (as NASA did with the O-ring erosion problem) is not the right response. It reminds me of a case in the building of the 688 class submarines: a cracked weld, that had supposedly passed multiple inspections, was found in part of the torpedo-handling equipment. At that point, all welds, including pressure-hull welds, became suspect.
It can't even handle intersections.
Ignoring some stopped objects is only a temporary limitation.
That may or may not be a temporary limitation. It depends on whether vision processing becomes reliable enough where you can almost always distinguish an object in the road from an object in the background.
And I'm saying that even that is incorrect. Here is a Tesla failing to stop: https://twitter.com/greentheonly/status/1134987596508143617 - The driver clearly noted that aiming for a less clear spot (dangling wires) triggered a stop.
Here is another example, in which greentheonly has used root to overlay what the car sees: https://twitter.com/greentheonly/status/1135011184439169024 - It sees the truck but is treating some of the space under it as drivable space.
Another view, with more details on what the car considers drivable vs non-drivable: https://twitter.com/greentheonly/status/1134490014321131521
Also, a clear case where it did stop: https://twitter.com/greentheonly/status/1134489977801334784
The Tesla system has limitations around object identification, but absolutely does NOT just filter out non-moving objects.
Humans have weaknesses too. What do the data say on their relative performance? I’m not taking a position here, just asking for data.
Its about as easy as driving gets. This is why a level 2 "autopilot" can parade around the place as something that is both innovate and safe.
None of the data compares what happens at junctions, which is where the majority of crashes happen.
It cannot work in suburban sprawl, It can't cope with poor weather, it can only drive on highways.
I was fully engaged and ready to take over, but never needed to.
Until traffic on the other side of the median barrier hit a big deep puddle, absolutely covering my car with a thick sheet of water. Autopilot immediately started screaming at me to take control. I already had my hands on the wheel, but it still scared me because I couldn't see either! After the water washed a way in a couple seconds, I was able to re-enable AP and keep going.
“Cars driven under traditional human control are currently involved in approximately 1.18 fatalities for every 100,000,000 mi (160,000,000 km) driven.[3] According to many automotive safety experts, much more data is yet required before any such clear and demonstrably higher levels of safety can be convincingly provided”
Anecdotes:
"I might be too drunk to drive, but I'm sober enough for my Tesla to drive."
And every Tesla driver I see engaged in two-handed cell phone operation with no hands on the wheel.
How is it not obvious that FSD most likely requires artificial general intelligence and is, thus, probably somewhere between "many decades away" and "humans will never achieve it"?
If you're accustomed to horses and see someone selling cars, it'd be reasonable to say, "Those are ridiculous. There are numerous places only my horses can reach. There will virtually never be a time cars fully replace horses." And you'd be right. It's still true--there are places horses can reach that cars cannot. But because of the value of cars, society has reoriented itself such that reaching those places doesn't matter to the vest majority of us.
I think self-driving cars will be the same. We don't need them to operate in every single road / circumstance ever. There may always be roads that self-driving cars can't drive on. But it'll become irrelevant because the vast majority of places people want to go will become navigable by self-driving cars, whether through the cars getting better or the roads becoming more easily navigable.
40,000 traffic accident deaths per year in the US.
< 400,000,000 drivers.
So more than 1 fatality per year per 10,000 drivers. There's a 99.99% change that you won't kill someone while driving in a given year.
That's just fatalities.
> In 2017, there were an estimated 2,775,608 police-reported motor vehicle crashes in which at least one passenger vehicle (i.e., a passenger car or a light truck) was towed from the crash scene in the United States, which resulted in an estimated 1,686,240 known passen-ger vehicle occupant injuries.[0]
So 0.6% per driver per year.
So far today I've seen one driver going the wrong way down a one-way road, and one run a red light. (Plenty entering the intersection on yellow and leaving on red, but at least one where it was blatant). I was on the road for maybe thirty minutes.
Do you think the average driver spends less than five minutes per year driving unsafely?
[0] https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
OTOH, if it means that the vehicle is liable to have a 25-second "blackout" in the middle of your morning commute roughly once a month, it seems very different.
These writers, being tech writers and not AI experts, didn't understand the subject very well and basicaly just repeated the industry's claims without really understanding the state of the art.
Then people reading those articles who also were not AI experts and didn't have much of a technical background anwyay (except being interested in cars) got taken in, thiking that if the press says that self-driving cars are just around the corner, then they must be just around the corner.
Why did software folks in particular got taken in? Because most people who write software don't write AI software and even many who use it don't understand it. Computer people are used to thinking of computers like magical dream machines, so the common reaction to overinflated claims about AI in general is wonder before surprise and cheering before skepticism.
Did high accuracy OCR without AGI seem possible before it was developed?
It was always known that pattern matching would be what ML algorithms would be great at - since the 60s or do. Reasoning about and "understanding" things - well, not so much. Some people hoped that this behavior would at some point just "emerge". But they basically resorted to this because they did not see any promising path to these goals.
- People with a vested interest in FSD being RSN playing a game of topper with each other, investors, and the market
- Extremely rapid progress in supervised learning, especially vision, being extrapolated into thinking along the lines of "How much longer can getting to FSD really take if we've made this much progress in just a few years?"
- A bunch of young urbanites with dreams of never having to buy a car or even learn to drive bought into all the hybe because they so wanted to believe. (And no small part of the backlash is probably that many of the same group feel basically kicked in the teeth at this point as reality sets in.)
I don't see why self-driving vehicles can't be dependant on additional self-driving-vehicle-related-signage or sensors embedded in / around / near roads, etc etc.
Humans are fairly dependant on additional signage, maps, and navigation tools (Google / Apple Maps).
If we can accomplish such a shift, there are already much more worthy ends to put the money and political will towards: increased public transportation, rebuilding of streets to safely support non-vehicle traffic, public ev infrastructure, etc. Spending massive amounts to subsidize the few rich who could afford autonomous cars would just be further regressive policy.
Because it doesn't. The issue is that all the self-driving car companies ignore how people actually drive.
People memorize their local driving area--it's why teenagers are so incredibly dangerous and why if you put a new stop sign into an area you cause chaos for several months. A self-driving car needs to do the same--the advantage is that a self-driving car can download that "memorization" for any new area it goes into.
Between memorized zones, people mostly "keep station"--they get onto a freeway and keep a relative position with respect to the lanes and the other cars on the road until they get back off.
When something goes very wrong, people panic and random chance takes over the fate of the vast majority of people.
None of this represents general or artificial intelligence. However, solving these tasks requires a lot of very grubby, non-scalable work over a lot of grubby data.
Collision Damage Waivers and the fact that they are relatively expensive suggests the opposite.
Accidents near an airport are more than 50% of auto accidents I have had totally in my life. All of the airport accidents I was literally not moving when someone drove into me.
We vastly overestimate the competence of the average driver.
Self-driving car companies have been memorizing ("mapping") areas for years.
Waymo car fully self driving three days ago:
Now this felt like a student driver, overly busy with correctness, insensitive to the actual experience.
I think these cars should be marked, like student drivers, to signal their inevitable difference in behavior.
Ideally we should not be accepting either. It is outrageous how many people we allow to be killed regularly for slightly more comfortable transport.
Slightly more comfortable than what? Horses? Bicycles? Walking?
There were 158,493 registered deaths in Australia in 2018, [1] of which 1137 were due to motor vehicle accidents [1].
Motor vehicle related deaths don't even rate in the top 20 causes of death. [also 1]
The rate of motor vehicle related deaths in Australia peaked in 1970 at 30.4 per 100,000 population per year, and has been on a steady decline since and now sits at 4.6 per 100,000 per year.[2]
This seems reasonable to me.
1. https://www.abs.gov.au/AUSSTATS/[email protected]/0/47E19CA15036B04BC...
2. https://en.wikipedia.org/wiki/List_of_motor_vehicle_deaths_i...
According to [1] in the US road traffic accidents are the 17th-most-common cause of death across all age ranges.
But among 15-24 year olds? Number 1 cause of death. In fact you have to reach age 45 before road traffic accidents are even outside the top five.
If you really want to ramp up the rhetoric, compare road deaths to mass shootings and September 11th. Where's the trillion-dollar invasion of Detroit to overthrow the Drivetrain Of Evil?
[1] https://www.worldlifeexpectancy.com/usa-cause-of-death-by-ag...
Well, if we are going to do that, we need to compare the utility of those mass shootings compared to the utility of a car.
It should really be compared with other accidents. Cyclist deaths, bathtub drownings, slipping on banana peels. Choking on food perhaps. Have the feeling it would rank pretty high in its cohort.
> Slightly more comfortable than what? Horses? Bicycles? Walking?
Buses, trains, trams, scooters, many other ways.
71% of Aussies live in major cities, and most have no need for a car. Yet we are one of the most car addicted people in the world. Billions wasted every year on this. Ridiculous.
> for slightly more comfortable transport
In some large metro areas, maybe. Outside of that bubble millions of people depend on a car to even be able to work.
By singling out an economically and socially productive segment of rural communities to attack, he is in fact attacking those communities as a whole.
(Incidentally, it's not just a matter of food. Where do you think the concrete, asphalt and timber cities are made out of comes from? I've not seen many quarries or forests in major cities. Rural communities as a whole, not just farmers specifically, make urban life possible.)
Personally, I view a city that is safe from cars as the greater freedom. It means my children can walk to school/friends/store without the very real danger of being killed. Cars are the number one cause of death for children.
We have an urban growth boundary now which tries to limit further sprawl, but it can't change the fact that the Portland metro area is 2 million people most of whom live in single-family homes, or maybe low density suburban apartment complexes.
DC, where most people drive, averages about 70 minutes.
Commuters in Dallas, where everyone drives, spend just 54 minutes a day commuting: https://www.chron.com/news/houston-texas/houston/article/Hou...
Sometimes transit or walking simply isn't an option, and in those cases I drive. But given the choice, transit might take a bit longer but it's much lower stress. If a kid jumps out in front of the bus and gets run over, that would ruin my day but not my life. If the same happens when I'm driving a car, that's life altering.
There are lower-stake considerations too: the money I spend on bus fares would doubtlessly otherwise be spend on parking; or I would need to schedule enough time to hunt for free parking which just adds more uncertainty and risk to my day.
But personally, I prefer to take the subway and read and relax for half an hour before and after work, instead of having to drive in rush hour.
You _can't_ drive to central paris. London you're looking at a £15 charge and a time penalty of ~15-45 minutes.
We seem to accept being killed by someone else, since that is how most of the 40,000 people are killed every single year in the US. I don't hear too many drivers complaining about that.
Tomorrow the situation will be different. But it's hard to trust such a broad safety claim knowing it's based on partial data and say "the failure rate [for self driving] is on average /better/". Comparing failure rates obtained is such different conditions is misleading.
Self driving cars are like a student driver with supervision. They currently drive in relatively controlled environments, or at least ones with very forgiving conditions, and are learning. And just like student drivers with supervision they are relatively safe. Until the moment they're exposed to the full set of driving conditions without supervision. We intimately and instinctively know and trust how the learning process work in humans. For cars not so much. Not yet anyway.
As a driver, that's not really something you have control over, apart from driving defensively. Your AutoPilot Tesla can still get T-boned in an intersection by some guy running a red.
If I get into a car accident, my insurance company gets the fun job of litigating. Either I or the other driver will be found liable.
If the car was driving, however, and it makes a mistake then the manufacturer could be held liable. And since car manufacturers have deep pockets, you can bet they'll get sued.
This is why manufacturers are pursuing perfection. Not because of a moral obligation, rather, a financial one.
One bad driver kills themselves and maybe a handful of others. One bad software update kills... everyone. So, the demand that the software be near-perfect is not at all unreasonable. It’s strange not to acknowledge this fundamental difference.
> One bad software update kills... everyone.
That's a fair concern. If you push a software update out to a hundred million cars at the same time which causes some of them to crash into oncoming traffic at high speed, you could have a large number of fatalities by the time you identify that there is a problem with the update.
But phased deployments do mitigate that concern, because it provides time to notice the problem before the bad update is more widely deployed.
That doesn't imply "treating humans like beta testers" because systemic risk is a real concern even when you have thorough internal testing. Even a thousand test vehicles will not encounter the full set of conditions that hundreds of millions will. You would expect to see a flaw that occurs for one vehicle out of 2000 occur 0 times on a fleet of 1000 test vehicles but 50,000 times on an installed base of a hundred million. Having a total of 50,000 fatal traffic collisions in one day is totally unacceptable, but having 10 on a bad day would mark a significant improvement from the status quo.
There was no such suggestion. You can do thorough internal beta testing and still not uncover every bug that will surface in production, because encountering every possible scenario that will occur in a complex system following mass deployment is only possible following mass deployment, and it is still worth reducing the impact of those bugs.
> you are assuming that your phase-deployment mechanism always works correctly.
It can work correctly 30% of the time and still be 30% better than not having it, because then a serious problem has to coincide with a failure of the phased deployment in order to have widespread consequences. This is why safety-critical systems are layered, so that you don't need them all to work all of the time as long as having all of them fail at the same time is sufficiently rare.
So with a self-driving car, you have the mental hurdle of asking people to accept the risk of the car killing them in a situation they could likely have avoided had they been driving, even if it is statistically less likely than them killing themselves while driving.
The delay is mostly due to a lack of suitable EV's and suitable tech. If big business was ready, big business would ram this through as soon as it could generate a good profit or good PR.
1) Arizona pretty much invited this problem through publicly denying oversight or responsibility. This is more a problem for Arizonians than it is for self driving tech. If things were done above board, there would be more questions and concern.
2) Everyone is used to Ubers driving badly, nearly killing other people, and generally escaping the law. This wouldn't be tolerated from Waymo/Google.
3) Deceitful PR with the camera footage painted this as solely the pedestrian's fault and not an alarming sensor/software issue.
I agree. This isn't about beating the national safety average, which includes drunk driving accidents, inclement weather, motorcycles, old cars and unwieldy trucks. It's about providing a safer alternative in scenarios where you have no other. And, it probably needs to be affordable too.
There is a lot of other low-hanging fruit to which we apply vision technology (like early cancer detection!), it's too bad investors are climbing over each other to throw money down the long, narrow corridor that is self driving.
I don't consider that bad. Anyone is free to make bad investments. I do consider it bad that they haven't yet had to book that many losses, due to the hype still going strong and them finding enough other idiots to join the party. I'm longing for the day on which bad investment decisions (such as in autonomous vehicles, at least if you expect them to be just around the corner to save your business model) seriously wipe out huge investments again - as it should be.
You don't find it unfortunate when capital is allocated towards failing business models rather than tech that has real promise?
The actual problem is not that investment errors are made. Such errors will always be made, as long as it's impossible to predict the future. The problem is that it appears as if right now, we are in a situation in which the negative consequences of such errors can seemingly be postponed virtually forever into the future. Or at least that has worked great for a decade now (first cracks are appearing lately, but it's far from enough yet).
Anything in an airplane is automated: takeoff, flying and landing. Pilots make mistakes all the time, but they still need to do things manually to keep their practice high.
People just do not want to go in an airplane without any pilots. In metro's it's not that obvious if someone is driving it or not. But airplanes, 'we' want a human pilot to be in control, else we feel unsafe. I think the same can be said about cars.
With airplanes there's a whole system in place to make sure that every airplane essentially gets its own private road. This simplifies the problem of automation immensely.
So maybe I'm wrong, but I don't see how autopilots couldn't be programmed to handle most kinds of failures (engine failure etc). Unless the failure is with the autopilot itself of course (which could have backup systems)
Most of the accidents with airplanes are caused by humans (https://www.boeing.com/commercial/aeromagazine/articles/qtr_...). So the real question is if automatic recovery would still be incapable of offsetting all the human errors.
For a prime example, see US Airways Flight 1549. The pilots Chesley "Sully" Sullenberger and Jeffrey Skiles intentionally disregarded part of the official procedure for handling a dual engine failure because they realized it wasn't suitable for their current situation.
FAA Part 121 flights are already so safe that trying to improve safety by taking humans out of the loop would be nonsensical. In some recent years literally more people in the US have died by falling out of bed than were killed in commercial aviation accidents.
Maybe these companies just want it both ways. They can then brag about how safe their AI is, but blame the ignorant masses for it not actually being deployed.
At this point the ignorant masses are pretty astute to see beneath the hype there’s a disturbing amount of reckless optimism.
Which has not been true thus far. That's the problem.
> The driverless bar for safety is way higher than very clever driver assistance or even rocket science.
Why shouldn't it be higher? Why shouldn't it be on the level of airline travel?
In all fairness, commercial airlines operate in an environment that's mostly highly regulated with respect to maintenance, training, traffic management, etc. But I agree with your general point. After all--except arguably with respect to some drug side effects--there isn't really any consumer-facing product that regularly kills people and we shrug out shoulders and say "Hey, stuff happens" and get on with our day.
Legal "drugs" and unhealthy foods are something of a special case. They've been around for a very long time and we've decided as a society we're mostly accepting of them and have greater or lesser restrictions on their purchase and use.
But we're pretty unaccepting of consumer products that are assumed to be safe and then blow up or malfunction in some manner that kills a user minding their own business and not doing anything wrong.
Who said it shouldn't?