Most people wouldn't instantly know how to handle those cases either. Many people would obey the drunk guy. Maybe that's the right thing to do.

None of those are edge cases because every time it drives it will encounter some novel edge case that has never happened before and it will have to perform better than a human.

That's why these things could never be rule based, there are too many small exceptions. When they're not overfitting/able to memorize all your training examples, neural nets learn heuristics, just as people do. Different people learn different heuristics. Granted, they don't have much of the same context about the world that people do, it will take a long time to build enough examples for them to infer all of that. But Tesla's fleet is getting more driving experience every day than you will get in your entire life, and every time they train on one of those exceptions, the entire fleet will benefit.

Don’t even get started with liability. Once you take away the steering wheel the manufacturer is on the hook for every single mistake and every single accident. You’d be insane to be a manufacturer and sign up for that.

If drivers no longer carry their own insurance, this is probably going to be handled by insurance at the manufacturer level, and baked into the price. The insurance will demand certain processes to prevent large-scale bugs being rolled out.

I don't see any fantasy here, just a lot of work.

Humans can reason about things that haven't to them happened before. Today's machine learning systems cannot. As you say, to react appropriately they must have been trained to do so using human annotated data.

The argument against FSD is that you would need an infinite number of annotated examples, and an infinite number of subroutines for behaving in any identified situations, because the space of driving is effectively infinite.

Until machines are able to do general reasoning about things they've not experienced before then FSD is not happening. By the way, Demis Hassabis thinks that this sort of transfer learning is the key to solving AI.

That’s not how machine learning works. You can “train on exceptions” as much as you want, and you have 0 guaranteed results. It can help, it can make no change or it can cause unexpected regressions.

Media (and surprising amount of tech people as well) tend to claim that ML learning is like human learning - repeat something enough times and you’re done, you know how to do it. ML is no where close to that point.

Prove they are getting better. They run into the sides of trucks and off ramps on the freeway quite often (and don’t you dare blame the driver... it’s “full self driving” remember?)

Can you prove a machine learning algorithm does the right thing in novel cases it hasn’t encountered before? Nope.

And also, don’t reply with “well can humans”. That is a lame rebuttal. Computers will be held to an almost 100% non-failure standard before society accepts them. And that will never happen because of, well, reality.

I haven't done the math, but time spent on forest service roads, wedding parking lots, and boarding ferries is quite low overall. And the idea of driving in those instances doesn't bother me. Having the car take care of the other 99.999% of my driving live is what I do care about.

> None of those are edge cases because every time it drives it will encounter some novel edge case that has never happened before and it will have to perform better than a human.

If most of your driving involves drunks directing traffic, forest roads, and dirt-lot weddings then a self-driving car is likely not for you.

Which is all well and good until you realize a few things:

1) that “last 1%” happens every trip at any time. You will always encounter an edge case the machine cannot handle. Period.

2) as a result you have to always pay attention in order to immediately take over

3) you can’t because you (the royal you) are three sheets to the wind plastered drunk.

Sorry. If I have to pay attention for that 1%, it ain’t full self driving. And anything that encourages you not to pay attention 100% of the time is unsafe and shouldn’t be allowed in the road. And if I have to pay attention 100% of the time in order to take over, what the fuck is the point?

Take Japanese highway system. It is well maintained, has “rest” stations where you can park car for free.

If you could get an app that can drive drunk salaryman, or not even drunk just tired from Tokyo interchange to closest rest area to wherever.. you will win. Nobody will buy a car without that who uses car for highway driving, period. It is a killer app. Get off work at 9pm Friday, get the car to the IC punch the destination, wake up at a rest are 20mins from Ski resort, Onden, parents house etc

Snow, taiphoon coming whatever. Park at closest rest area.

I am very pessimistic about cars without steering wheels. I am quite optimistic about cars that have ability to drive well marked roads. Here is a crazy thing Charge fairs for highways like Japan does, then maintain them.

Yes there can be a big hole in the road anytime, the AI has to watch the vehicle in front of it, if no such vehicle then it should chose a speed that let's it evaluate road conditions for the given weather/visibility.

Vehicle coming into our lane? The AI has to match human level maneuvering to evade the incoming car. It already has much better chance given it won't panic, will be always fully alert, and will be as accurate and precise as it can.

So the big categories are sudden road/environment changes (tree falls on road, hail, mudslide, earthquake damages road, animal crosses road), other vehicles, and pedestrians/cyclists/etc.

All are manageable with inferences from the environment (weather and roadside context determine visibility and how much space there is for maneuvering, how likely are unexpected crossings - eg. deer, kids) and surrounding traffic.

Are these hard? Sure, but none require human level cognitive reasoning.

Are you are going to randomly have to park in a wedding lot? The only example you gave which might happen in the middle of a trip, the Tesla can handle just fine for long enough to pass control over to the driver.

> you can’t because you (the royal you) are three sheets to the wind plastered drunk.

It's still the driver's responsibility to drive sober. Even so, I'd far rather someone who is drunk be behind the wheel of a self driving car than otherwise.

Either you're driving, or you're a passenger in a car with a driver who seems reliable, but really isn't totally so. Eventually, that will become a winning bet, but when?

Driving safely on frozen surfaces is not a solved problem for human drivers, but most of us insist it is a perfectly reasonable thing to do.

If the vehicle drives as slowly as it should in those conditions, it would probably frustrate a lot of people who really depend on their false sense of invincibility.

A car has access to all 4 wheel sensors independently, it can apply brakes on each 4 wheels independently. It can always turn the steering wheel in the right direction, and it wouldn't panic.

Also, it would always drive the 'right' speed limit... sure, other human drivers might get annoyed, but assuming the true 'full self driving' future happens, there shouldn't be many of them on the road in time anyways.

What does that mean? That it actually is solved by humans? Or that it's not, and we just take the increased risk? If the former, then we can automated it. If the latter, then the self driving cars can also drive at increased risk, opt-in of course.

If we built planes that were only as safe as highway driving, people would be outraged.

We aren't exactly rational about this stuff, and we expect a lot more from machines we don't directly control.

But I like your opt in solution. Maybe the UI loudly complains about the risk of current conditions and sticks to <5 MPH, unless the user enables "never tell me the odds" mode.