Edit: not to mention state-of-the-art sensor fusion, with atmospheric composition analyzer, thermometer, multiple high resolution vibration sensors, stereo microphones and an advanced vehicle to vehicle signalling system backed by heuristic fallbacks.
One or two fixed position monochrome, low resolution forward facing cameras such as in a Tesla just won't compare.
I'm not saying it can't be done, but the Tesla engineers have a much harder task than if they'd had human level sensors. The Tesla "brain" is more like a human looking out of a tank through a periscope in a tank or something... and the periscope can't even be turned.
I'm pretty sure humans could cope with that. https://www.youtube.com/watch?v=-CITIXlw_T4
(3-camera depth is more reliable than 2-camera. Many of the ambiguous situations for two cameras can be resolved with three. Especially if it's 3 cameras in a triangle, not a line.)
Monocular depth perception w/out motion is a thing too, e.g. [1], however I doubt it is good enough for safety-critical systems like self-driving cars.
There's a great presentation from Gabe Sibley (now with Cruise) on camera based SLAM, or "Mobile Robot Perception for Long Term Perception in Novel Environments" that explains quite well the principles underlying how it works.
Gabe concludes at the end that probabilistic perception and modelling using cameras isn't reliable enough and that Lidar is likely necessary for safety critical systems such as autonomous vehicles:
If you can't detect rain with reliability, you're not ready for full self-driving.
Apparently Musk heard it (or something similar), because Tesla is rolling out the first self-driving features in August: https://news.ycombinator.com/item?id=17282006
Self-driving features? Sorry, there are no some self driving features. You either have full self-driving or you don't. If you don't (which Tesla doesn't), how the hell can you call it self-driving?
In any case, it's quite easy to imagine a plausible meaning "some self-driving features": perhaps these features enable the car to drive itself [without oversight] in some, but not all situations (<i>e.g.</i> on highways, but not in cities).
Minute point of irony, Musk seems to act more and more like geohotz which he criticized.
If this guy was working on adtech, that would be fine. That's very error-tolerant. But this guy is working on automatic driving.
The basic mindset here is to run image classifiers to classify the objects in an image, then use the classifier output to decide what to do. There's no geometric analysis. That's scary. Classifiers just aren't that good. See the earlier article today about adversarial attacks on classifiers. Classifiers pick obscure details of images and use them to make decisions. Nobody seems to know yet how to prevent that. This problem shrinks with larger data sets, where hopefully the irrelevant details cancel out as noise, but, as the speaker points out, that breaks down when you have few training cases of certain situations.
The Google/Waymo approach is to get a point cloud with LIDAR and radar, profile the terrain and obstacles, and figure out where it's physically possible to go. That's geometry based. In parallel, a classifier system is trying to tag objects in the scene, which feeds into a system which tries to predict what other road users are going to do.
With that approach, a classifier result of "not identified" is fine. The system will detect and avoid it, or stop for it, and make conservative assumptions about its expected behavior. Chris Urmson, in his SXSW talk, showed video of a woman in a powered wheelchair chasing a turkey with a broom. This was not identified by the classifier, but it was clearly an obstruction, so the vehicle stopped for it. That's essential here. It has to do something safe with unidentified or mis-identified objects.
At Tesla, Musk insisted that this could be done with a camera alone because humans can drive on vision alone.[1] So Tesla has people trying to make camera-only driving work. Not very successfully so far.
"November or December of this year (2017), we should be able to go from a parking lot in California to a parking lot in New York, no controls touched at any point during the entire journey." - Musk, in April 2017. This guy is saying what Musk wants to hear.
[1] https://blog.ted.com/what-will-the-future-look-like-elon-mus...