I understand why they made the "no-LIDAR" bet early when the LIDARs were completely unpractical for a production consumer car

However, nowadays it starts to look that 100% reliable depth estimation from cameras might actually require a human-level AI to work and also solid-state LIDAR technology is becoming cheap enough and integrateable into normal cars, but Tesla can't really change their stance on this without admitting that FSD options they already sold would not actually become FSD within the lifetimes of these vehicles. I suspect this might also be the reason why Karpathy looks more and more nervous with each new talk

> FSD options they already sold would not actually become FSD within the lifetimes of these vehicles

That's pretty much a given at this point but they will not admit it until a class-action lawsuit forces them to.

But we might actually get a functioning Mars base out of this since that's where Musk will be hiding when the FTC finally wakes up /s

>100% reliable depth estimation from cameras might actually require a human-level AI to work

you don't need 100%, and even humans are far from 100% (500Mpx resolution of our eyes allows to basically sheer brute force through it in many cases). The stereo setup provides great and fast estimation with several megapixel resolution with good fps (way better than lidar) for majority of situations. It is some share of the [part of the] scenes, and you really know it right then and there, where you need AI and/or very sophisticated compute heavy algorithms. So instead of throwing AI and the compute power at those parts, you just pull the points from the lidar (and even radar if the things are that bad) covering that segment. And that way, given a couple more iterations of sensors (from current 20Mpx+ to the hundreds Mpx) and compute, it will be doing even better than humans. Anybody not doing sensor fusion would be a loser though - just like going into a fist fight with one hand intentionally disabled.

Is it something that can be retrofitted on existing models?

LIDARs unlikely since they require a line of sight forwards, backwards and possibly on the sides of the vehicle

One of the gains from using lidar is also that it's a different sensor altogether from cameras, with different failure modes.

For example, cameras are sensitive to glare from reflections (sun near sunset or reflecting on metallic objects) and oncoming traffic at night. Lidars operate on a different narrower wavelength and are unlikely to be affected by that, although they might struggle with objects that have low reflectivity at a long distance.

The fact that these sensors are different means that the intersection where a dangerous situation would not be detected by either sensor is much smaller than any sensor individually.

In any case, once AVs are deployed at scale, if it becomes apparent that some sensors can be removed or replaced by something else, then they will be if there's a case for it.

> I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.)

On what set of metrics do you think Waymo is safer? IMO it's too early to compare and cherry-picked proofs both from Waymo and Tesla are not really representative.

For starters, Waymo reported 1 disengagement per 29,944 miles driven in 2020 to the California DMV [1], while in the talk, Karpathy implies that a Tesla being able to drive around the SF area for 2 hours without a disengagement is unusual. Note that Tesla didn't file a disengagement report because they didn't do any autonomous testing on public roads in California in 2020.

There are issues with reading too much into disengagements, but there certainly seems to be a large difference here.

[1] https://thelastdriverlicenseholder.com/2021/02/09/2020-disen...

Waymo's number for 2019 was around 11,000 miles driven per disengagement. It's been improving steadily, at a reasonably good rate.

Tesla wimps out and won't test in California, because they'd have to report.

Isn’t Waymo only driving in small HD mapped areas? Previously the were only driving within a 50 mile area of Arizona where it’s clear weather all the time.

Waymo has published detailed safety performance data of their Arizona operations: https://waymo.com/safety/performance-data

You can read their other safety whitepapers in https://waymo.com/safety

Arizona roads are also mapped to extreme precision, have very wide lanes, and are optimized for cars. Waymo has prioritized low intervention by being overly cautious and avoiding hard maneuvers (like many left turns).

That doesn't work when they scale up to any other set of normal roads, especially as density and complexity increases.

They don't avoid left turns. There are plenty of videos from Chandler, AZ of Waymo performing unprotected left turns perfectly fine.

They will always map roads to precision, whether it's Arizona or San Francisco. Why is that a problem? You should either look at their CA disengagement reports over the years or wait until they roll out a service in SF (where they've been testing heavily). That will show how safe they are in dense environments.

From what I gather, they manually mark sections as hard when the cars get stuck there, e.g. due to road work, and then their routing system chooses another route, e.g. one that avoids the left turn.

The video with the Waymo car getting stuck and taking off from the rescue team had an example of this.

I guess it makes perfect sense from a engineering perspective.

Definitely. Especially with car companies like NIO strapping in LIDAR to their upcoming models.