I agree, being better than human is a sales point that I expect to see in brochures. One of the ways I would expect that plays out is self driving transport cars for high value targets like world leaders and drug lords. "This car will respond faster, and more accurately, to get you to safety before a human driver even knew there was a problem."

That said, John stated that without LIDAR you couldn't adequately meet the environmental challenges and achieve good self driving.

Specifically "I'm still not happy with self-driving on vision alone ... There are too many hard cases for vision." which boils down to a disbelief on the imaging processing side of the pipeline where NVidia has been attacking using GPU type architectures to extract image information rather than generate it.

One way to evaluate how far the image processing pipeline has come is to look at research on how well it can classify images. And in that space, in the research, it is doing better than humans [1]. As I've said elsewhere I think LIDAR was a crutch that worked well to cover for weaknesses in classifying images, but I recognize that the crutch may no longer be needed (certainly Tesla and Nvidia are trying to make that case).

[1] http://www.eetimes.com/document.asp?doc_id=1325712

There is only a little evidence current image classification models outperform humans. The 5.1% number is just the number from one grad student who went through some ImageNet images himself - e.g. a single blog post (http://karpathy.github.io/2014/09/02/what-i-learned-from-com...). There really hasn't been a concerted effort to see how humans actually do on ImageNet.

I am willing to believe that a group of humans trained on ImageNet and without time constraints would be able to outperform the state of the art neural net models.

> One way to evaluate how far the image processing pipeline has come is to look at research on how well it can classify images. And in that space, in the research, it is doing better than humans.

Well, computer vision is doing better than humans on the ImageNet dataset, that does not mean it is better than human on a driving image dataset.

> One way to evaluate how far the image processing pipeline has come is to look at research on how well it can classify images. And in that space, in the research, it is doing better than humans [1]. As I've said elsewhere I think LIDAR was a crutch that worked well to cover for weaknesses in classifying images, but I recognize that the crutch may no longer be needed (certainly Tesla and Nvidia are trying to make that case).

I have worked on systems for automatic target detection and classification. Systems have a long way to go before they reach human-level accuracy in classification. Even on "artificial" images like radar return maps humans are still slightly better, and many systems pass the underlying processed data to the human operator for final review. Tracking and classification in a dynamic environment is hard. It's even harder when trying to rely on passive sensors to do it.