I don’t agree he makes good points at all. His arguments essentially come down to two things:

1. LiDAR is expensive - this makes sense from his perspective because he has already declared all Tesla cars on the road are capable of “full” self driving with their current hardware. If he walks back on his, he would have to retrofit a million cars.

2. Humans can drive with just vision, so why can’t computers? This is a naive argument because human vision isn’t just vision, it’s linked to a ridiculous intelligence in the form of human brain. And computers are nowhere close to replicating it.

If you want to read more on LiDAR’s importance, read some of Brad Templeton’s writings.

https://www.templetons.com/brad/robocars/cameras-lasers.html

https://www.forbes.com/sites/bradtempleton/2019/05/06/elon-m...

I doubt the cameras on the Tesla have anywhere near the performance of the human eye. I own a Tesla and often I get some warning when the sun is at a low angle blinding the cameras. I've no problem seeing stuff at the same direction.

So both sides of that sentence seem wrong.

EDIT: it's also worth noting that Tesla is accounting for a whole bunch of future revenue from FSD. If Tesla admits at some point that they can't make it with the current sensors that would have huge financial implications.

I agree that the second point is overly simplistic, I do think there is some sense in the argument and that it remains to be seen whether the state of AI will advance enough to be better than humans. Yes, humans do have intelligence and that does help quite a bit, but I would also guess that a system could potentially be better than most ordinary human drivers without LiDAR just because they have a much better view of all cars around them (humans have a somewhat narrow field of view and can only look in one direction at a time, computers can have a full 360 degree view all of the time). They never suffer fatigue and never get distracted. My guess is that, yeah, it is better to have LiDAR, but also we can also probably make a really good system that is better than humans on average and saves many lives per year without it. It seems worthwhile to me to continue exploring and investing in.

It is definitely worthwhile to continue exploring this. But as you say yourself, there needs to be significant improvements in the state of AI to do this. So Tesla seems to be relying on achieving significant breakthroughs to make it happen. I wonder if it's too risky of a bet considering they are already taking money for FSD from customers and promising the feature.

Also, I don't think Tesla is known for their AI expertise like Google. So it remains to be seen if they can pull it off.

Until they lose peripheral vision later in their lives, people's field of view is about 180 degrees.

I’m human, I use more than vision to drive, as in I actively think “what will the other drivers will do?”.

The more difficult form of the question is “wtf does that person think they’re doing!?” People are weird. It’s not enough to think at a human level in easy conditions.

You have to think at human level in corner cases, like seeing an object fly into view and judge whether someone is likely to be chasing it into the street, which depends on knowing what the object is and how it fits into people’s lives.

Also, blind humans (and bats) can walk (and fly) around the city pretty easily with just sound. Why would you tie your hands behind your back and try to make a robot that can navigate the city using only sound?

To cut corners and costs.

Point #2 is deeply flawed.

Not really a good example. Blind humans don't navigate cities easily, even when cities take them into account. As an example, crossing a street despite tactile surfaces on each side can be a hassle because it's difficult to tell if you're walking straight or diagonal towards the other end, potentially walking into a car. Sound only takes you so far.

Source: I have blind friends.

> 2. Humans can drive with just vision, so why can’t computers?

Agreed this is very naive, it implies that humans are good drivers. Humans are awful drivers, and it sure doesn't help that [human] field of view and depth perception is very poor [compared to alternatives such as LIDAR].

Human drivers cause ~40,000 deaths a year in the US. The car would not exist as a consumer product if it were invented today. Automatic car systems may have to have close to zero fatality rates to be acceptable in the US. Possibly a national law of some sort could be passed to limit automatic car manufactures liability that would allow them to be deployed at scale. Otherwise, I see little chance of automatic vehicles existing in the US. They will cause deaths, especially if human drivers are still allowed on the same roads. I sure hope they can be deployed, as the benefits of automatic vehicles for safety, convenience, the environment, and efficiency are large.

Step 1: Computer vision.

Step 2: Theory of mind.

Step 3: ???

Step 4: Profit.

well lets also take into account in the US alone over three trillion miles are driven and the fatality rate per hundred million miles traveled has been reduced over time.

So yes the numbers are high but the item that a car using vision would have similar problems is a bit dishonest and LIDAR only provides one other means to determine where something is.

However self driving cars have one major advantage that LIDAR and cameras don't solve. They are not easily distracted. They are also far easier to be restricted to the ability of the vehicle they are managing adjusting for weather and more. Oh you can fool them for sure and there are always edge cases.

Not my understanding of his premise - I don't feel like your statements have actually countered his point. I have actually been looking for a true counterargument to their approach and so far everything I've come across I don't buy into. If anyone has anything countering/disputing Tesla's approach - please share! I can't help but feel like there is something I DON'T see that many smart, respectable people see and understand that I just haven't seen.

Also thanks for the link shares, but I don't think either actually disproves Tesla's approach.

My understanding of the Tesla approach is: In order to truly 'solve' self-driving (situations on the road that have never been seen before to drive safely - think unannounced construction, collision or road closures due to protests), you MUST solve 'vision' with a very, very complicated and well trained neural net (re: ridiculous intelligence in the form of a human brain as you state). In addition, the existing road infrastructure (re: signs) is all built around human vision - and so being able to identify and interpret all of that is a requirement.

I find their approach compelling as in this instance where you have cameras with a particular neural net (which they are constantly refining the learning model on) that are training across the millions of cars across the billions of miles across the thousands of various edge cases into a generalized solution. You also have a re-enforcement loop via the nature of a human driver which 'intervenes' through the drive, a necessary step in refining the model at scale. Note: I am not saying that Tesla's FSD will be coming to a street near you anytime soon. BUT, I haven't heard or understood a well articulated argument that says 'lidar is really the only practical solution'.

This also doesn't factor in that I've heard Lidar doesn't work well in any kind of precipitation (light being refracted away from the sensor). Also, full self driving doesn't mean it can drive in situations that humans WOULDN'T be comfortable with (i.e. snowy blizzard or thick fog) so in either solution shouldn't be factored in as a part of the required solution set.

And finally, the practical threshold on a FSD system that would pass regulatory approval is evidence/data that it is materially safer than a 'typical' human driver. It seems the 'throw millions of cars and billions/trillions of miles at it' with a refined tagging system that approaches the narrow vision solution for driving forward in 'driveable space' seems to be most likely to reach a solution first.

> My understanding of the Tesla approach is: In order to truly 'solve' self-driving (situations on the road that have never been seen before to drive safely - think unannounced construction, collision or road closures due to protests), you MUST solve 'vision' with a very, very complicated and well trained neural net (re: ridiculous intelligence in the form of a human brain as you state). In addition, the existing road infrastructure (re: signs) is all built around human vision - and so being able to identify and interpret all of that is a requirement.

Yeah this is dubious. You need to solve situational awareness. Vision is one way of doing this. Lidar is another, and lidar avoids many of the drawbacks of vision (having to do accurate world modeling based on cameras).

Tesla doesn't (and would be stupid to) feed camera data directly into a neural network. They feed multiple cameras into a complex system that involves both classical object positioning and neural networks to build a model of their surroundings. Then a downstream system consumes that model and makes decisions.

Its not a single end to end black box. Such an approach would be computationally infeasible, not to mention over-parameterized to all hell. No one does this, not Waymo and not Tesla.

While cameras are good at certain tasks (like detecting traffic lights), they are not good at all tasks, and using more specialized hardware for object detection and world modeling means that lidar based systems are strictly better. They have more information than camera based ones.

Tesla is betting on, somehow, making some breakthrough in computer vision that no one else can replicate, and further that lidar can't do what cameras can.

Your argument appears to be that since Tesla has more data, they'll achieve some eventual success, but the point is that they'd achieve more success faster with lidar, and everyone in CV seems to agree that we'd need pretty fundamental improvements in CV (and perhaps in cameras) before you get the same performance out of CV that you get out of lidar. That means that Tesla's betting on a less accurate world model being good enough. Maybe they're right, but so far we have some evidence to suggest that cameras alone have some pretty fatal shortcomings, and no evidence that Tesla has solved them.

He has to make that argument because he has to justify not having LIDAR on Teslas, and he doesn't have LIDAR on Teslas because it's still too expensive to have it mass marketed. But as with most things, price will keep going down and it'll eventually become the go to, and Tesla will be stuck with years of camera data and zero LIDAR training.

Your answer just shows a lack of understanding what Tesla is doing. They have a layer that transforms camera data to depth data. So if they wanted they could replace that part with lidar and just cary on from there.

This isn't some CRUD app where they can refactor to use another class with a matching interface.

The models they use are incredibly opaque, complex, and work based on statistical inferences that are built into the data. Making fundamental changes like assumptions about where that data came from is not trivial. Even more so given that we're talking about safety-critical applications.

Tesla's model has difficulty generating depth data.

This has been demonstrated many times by Teslas crashing into stationary or large obstacle in front of the cars that would have been immediately apparent to a human driver or a car with LIDAR.

LIDAR alone isn't the best choice. Neither are cameras alone. They each excel at different things.

That's why everyone else in the industry use both types of sensors.

I watched the whole video you shared. I think they made the a mistake assuming that people are good at driving with just vision.

People are actually terrible at driving, even when paying attention.

I really hope that the future of self driving reduces the road toll to a teeny fraction of its current level.