I do think the "last 1%" may be a very hard problem to solve, but the majority of crashes are simply people not seeing things or making bad decisions. "Recognition Error" and "Decision Error" are the two most frequent categories of accidents which basically boil down to not seeing obstacles while performing a maneuver (changing lanes and not seeing a vehicle in that lane, turning and failing to see an oncoming vehicle) and driving too fast respectively. ( see https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...)

I don't see why the "last 1%" problem needs to be solved when we have hundreds of thousands of accidents every year caused by people not paying attention, not reacting in a timely fashion, and driving unsafely; all problems that computers are likely to excel at.

Because a half-baked product is dangerous (re: handover to human) and unsatisfying. You really want the version with no steering wheel. I’ll take the intermediate one, but let’s work toward the good one.

>the "last 1%" of self-driving software may be equivalent to solving AGI.

Exactly right. You need a semantic world model, reasoning and prediction. AGI bottlenecks self-driving, assistants, text understanding…

I think we are overestimating amount of intelligence we humans need while driving...

The edge cases are unclear: construction zone, heavy equipment, school kids, another car poised to enter the road.

They are probabilistically modeled already. You keep tracking other objects and compute various possibilities for their movement and depending on the past and present behavior pick the most probable one. If car encounters lower-than-threshold probability, it can always slow down or stop, like what non-reckless humans would do. Even end-to-end deep learning can recognize construction zones and learn to drive around traffic cones.

The edge cases are endless (wheelchair woman chasing a chicken), and without a semantic world model the car will have no idea what to do. Low-level object avoidance architecture isn't enough.

Actually that case would be detected as two objects on a collision course and car would stop/slow down. Mind that sensors scan surroundings with high frequency and some of them can see "beyond the walls" like radar. Ultrasonic sensors can also detect the state of road surface (dry, wet, icy etc.).

Yes, but when can the car proceed, and in which direction? Helps to know what a wheelchair, a woman and a chicken are and how they behave. Helps to be able to speak with them.

Superhuman sensors will be great, but you really can't dodge AGI for true full self-driving.

It's probably super easy for the car to stop and open the window for a person to talk ;-)

True, you could have the passenger making "shoo" noises :D