Google is at stage 3 and getting close to stage 4 for slow (< 25MPH) vehicles. See Urmson's talk at SXSW, where he shows Google cars handling many unusual situations. (A woman in a powered wheelchair chasing a turkey with a broom is one example.) If you're willing to drive slow, more situations can be handled. It's not necessary to have as much sensor range, situations are less ambiguous, braking hard will resolve most problems, and predicting what's going to happen becomes less important.

At higher speeds, prediction becomes more of an issue. Will that car on the side street drive into the intersection, or not? At slow speeds, you can wait and see what they do. At higher speeds, you have to predict behavior. That's hard, but a machine learning problem.

This is with the caveat that the weather is nice and you're in one of Google's specially mapped areas. As I understand it - and I never worked at Google and I keep waiting for someone who does to correct me on it - those maps are very difficult to scale and maintain.

The limited area/weather isn't a huge problem.commercially . You just need to start offering it in a single city when possible, prove the tech and business works , and than you could get tons of money to scale to other cities .

I don't see how that solves the weather problem, unless you're limiting yourself to cities with nice weather. Even then - all cities have bad weather occasionally.

I'm not arguing against progress. I think it's great people are working on this and obviously you have to start somewhere. I'm just pointing out the fully autonomous vehicle is not a few years away.

Part of the problem I think is that the image is not fully used; I mean, generally these systems consist of a black-box routine that extracts interest points and then passes it onto to a SLAM routine, which inturn keeps an estimate of the car state and the interest point positions. There is no "physical" model of the world being inferred from images, and I imagine this makes things rather tricky (and also why a LIDAR is so much more useful).

AFAIK deep-learning hasn't really brought much change to this manner of doing things - the mapping part esp.

I think you must mean that you don't know an academic expert who thinks we're close. The teams at Tesla must think it's achievable in the next few years.

Well I also used to work with the team that's now at Uber (I worked at the National Robotics Engineering Center which is where they got almost all of their initial 40 engineers), and some of the guys I used to work with went to Google as well as Tesla. Tesla has a retention problem - their top engineers regularly quit every few years. Google lost nearly their entire team working on self-driving cars because Sergey insisted on making a fully self-driving vehicle instead of rolling out incremental levels of autonomy.

If you're only reading article after article hyping the technology instead of talking to the people actually building it, of course you're going to believe the journalists instead of the engineers.

I don't know anybody actually writing the code that thinks we're close. Of course the PR teams and managers are going to hype everything up - that's their job.

Excuse my rather hand-waving explanations below but I'm not an expert in this field.

The guys you spoke to, was this before or after they went to Google/Uber? As I understand the approach at Tesla would be (a) collecting an enormous amount of real-world driving data that possibly others are not or have not done yet, (b) do as little "coding" as possible, but rather take a deep-learning approach. I.e. the "algorithm" gets better the more data you throw at it, it doesn't depend on human intelligence, but rather on how much data you collect. Similar to how Google Translate got so good (it's not good because of any linguistic model or because a team of linguists "coded" it, it's just good because of the sheer amount of data it was trained on). (c) they have enough money to throw at any hardware requirements for a platform that could train on such an amount of data.

Are the conditions above not different perhaps from what you experienced whilst working at NREC and perhaps different from the way you guys approached autonomous-driving? I'm trying to think from Elon Musk first-principles. If it was technically possible, what are all the lego blocks you need to go about building this?

That certainly seems like the idea for Tesla - collect all the data from the cars in real-world condition and use it for training. I think it's brilliant. Time will tell to see how well they can leverage that data. Having a better sensor suite would certainly make it easier though. It's a lot easier to interpret LIDAR (it basically makes a 3D map for you) than it is to interpret a camera (you have to figure out how to turn 2D into 3D).

My skepticism is in the leap towards a fully autonomous vehicle, safe for driving on real roads in harsh conditions. Very few people have attempted anything in bad weather yet, although Ford has actually done some work on it.

If we are talking about incremental improvements - sure that will happen constantly. What I'm saying is I don't believe you're going to get into a car without a driver anytime soon.

Tesla has been recording driving data, sensor data, and GPS data for years. They already have a system in place for machine learning driving data. This combined with their sensors is IMO a great basis for fully autonomous cars.

Tesla is planning to have their factory produce 500,000 cars in 2018. Even if they only meet half of that they'll still be collecting a lot of data.

I would guess most major cities and freeways will have full autonomous support by 2019.

I'll give you 10 to 1 odds on that if you want to bet.

Driving in snow is simple. Just hit the brakes hard whenever you are uncertain.

I think the comparison to google translate is interesting. How much confidence do you have that a translation by google translate is correct? How would you compare the complexity of translating documents between various languages to the complexity of driving a car in arbitrary environments and situations. And the most interesting think: Would you trust your life on google translate? At least in the sense that it does not corrupt the meaning of a translated text (small errors ok)?

The way I see it panning out is that the first generation of autonomous vehicles is going to have a tough time. As more and more cars become fully autonomous there will develop some sort of protocol where every autonomous vehicle will be relaying exactly what it's "about to do" to anything nearby. I.e. your car will know if the car in front / behind / on the other side of the freeway is about to do anything and thus eliminate what is currently seen as "unpredictable environmental factors" to a very large extent. Internet of Things will also increase the amount of additional data being fed to the car's computer, e.g. it will know exactly when a traffic light is about to turn red because they will receive this data directly from it.

Once you reach a certain critical percentage of fully autonomous vehicles regulation will probably change to such an extent that car insurers will void your insurance if you drive it manually, or give a significant discount to your insurance premium if you don't ever put it in "manual" mode, etc.

I think the problem is really difficult right now because humans are still allowed to drive, but as that ratio goes down it becomes less of a serious problem.

Or they want you to think that.

Or it's the marketing team and not the engineering team :-)

What do you make of this?

https://www.tesla.com/nl_NL/videos/full-self-driving-hardwar...

Researchers already passed human level image classification performance in 2015.

https://arxiv.org/pdf/1502.01852.pdf

That's a really cool paper. While I wouldn't say they're doing better than a human (I really think they're twisting semantics with that one), it is impressive and I hope to see a lot more progress in this area. Thank you for sharing!