A machine controlling a real car gathers feedback no faster than real time and cannot afford to learn the meanings of street signs from the consequences of ignoring them. A machine can learn the rules of Atari games from scratch by playing them orders of magnitude faster than real time and treating "death" as one signal among many.

In order for a machine to learn driving the same way it learns Atari games, it seems that it would need an extremely high fidelity virtual environment to learn in. The high fidelity requirement would necessitate a lot of up-front investment in trying to get the simulations right. You might spend a whole career just trying to build a drivable Virtual Philadelphia as challenging as the real thing. The details would also make it much more expensive to run training sessions at high multiples of real time.

Given those factors, I'm not surprised that self-driving vehicle experiments just use real environments and don't try to learn the fundamental rules from scratch. But it's an interesting point that these choices may make it harder for agents to keep improving.

There's a middle ground too which alpha go leveraged which bootstrapped learning from actual human gameplay predicting user actions. That's what comma.ai does and AlphaGo still performed significantly better than explicit rules or higher level abstractions. That and a mix of simulations might yield better results

If you could make a virtual environment that was that good, you would have already solved the self driving problem.

I wonder if Google would be willing to pay people to add cameras to their cars to collect real world data in far larger scale.

I do believe that's the basis of Tesla's play in the space: a bet that enough cameras collecting real-world data can beat out a dedicated from-scratch self-driving system in deployment.

Whether this will work remains very much an open question.

That might be happening now already. I think Google sourced their dataset from dashcam videos uploaded to YouTubr?

Really not true. We have all the tools available... Just model a city in Unreal Engine 4 (GTA 6 anyone?). Your sensors, like LIDAR could do ray-queries, cameras could render views, etc. It's not 100% real, but probably real enough to learn the basics. Photorealistic graphics should be sufficient for AI to learn real-world interaction. In the end our whole world might be just that, a simulation. So I see zero reason why we couldn't train a self-driving car AI in one.

And the coolest thing is that this could also be used as a basis for a AAA video game, and these tend to make billions as well these days, so it's a win win for everyone. AI companies with funding should invest heavily into virtual reality and gaming, because they will need to perfect this to train their models.

The problem is that these simulations are far from perfect, and AIs are great at exploiting paths-of-least-resistance. So you'd end up with an AI that could drive through your virtual city flawlessly by overfitting on some cues that you wouldn't even notice, but are reliable enough because of the necessarily lower complexity of the simulation.

It wouldn't translate to the real world; unless maybe you add enough noise to the simulation to prevent the AI from using too simple cues, but at that point, it's questionable how much insight the AI could still distill from the simulation.

In that case, I would like to see what the AI can learn in Euro Truck Simulator 2.

Minor tangent, but I think you’re referring to this video[1] with Dmitri Dolgov. He’s the CTO of Waymo, not CEO.

[1] https://youtu.be/P6prRXkI5HM