It is a gamble, but also betting on LIDAR is a gamble, as it could be a pit to fall in if it turns out not to be beneficial. The big benefit of LIDAR is, it gives you very good 3D data about your environment. Unfortunately, it completely misses a lot of information which is essential for driving and actually a lot of our traffic rules are based on things which you cannot detected with LIDAR: traffic signs, traffic lights and so on (you could detect the sign itself with high precision, but not what it reads).

This means, LIDAR could just be additional sensor data augmenting the camera information. But it adds a huge cognitive load when combining the data. Besides the costs of the LIDAR system itself, it adds to the computational costs. Once the 3D-data from the cameras only is "good enough", you might get better results by spending the available compute resources rather on improving the analysis of the visual data alone. And the 3D data Tesla seems to be getting from the cameras seems to be pretty good already.

The big elephant in the room isn't to recognize a road sign correctly, it is to decide, whether it applies to the car or a different lane, road or whether it is actually a burger king logo :). There LIDAR might be just a distraction.

It will be interesting to see how the autonomic car race will pan out. Especially, when competitors are using different strategies. That als ensures, that not everyone is going into the wrong direction.

Because that adds cost. The game here is producing a car that is cheap to manufacture that can drive itself without having to have a data center in the trunk and a large lidar bump on the roof.

The gamble here is on the software, not the hardware. All the accidents that happen are not a failure of sensors not sensing but of software not detecting what the sensors captured. We're talking cars crashing into things on a clear day. Cars not recognizing situations that a human would recognize from the captured video. Etc. They need to solve that anyway. Regardless of whether they have lidar or not.

Other companies like Waymo pushing the Lidar agenda are also behind on their schedules to launch autonomous driving. But at least they have actual driver less cars on the road right now driving actual passengers around in very select markets.

Either way, this is more or less a greenfield market without a lot of contenders shipping actual products yet. So, Tesla can afford to spend more time developing this market. If they nail this somehow, eventually, it will be a big growth driver for them.

Not sure if the cost and the "lidar bump" will still be arguments in the future. Heck, the latest iPhone has a lidar sensor integrated! Surely with a lot less capabilities than what you would need in a car - but it proves that it can be built small and cheap. And I would say that more data from sensors that sense different things will always result in better decisions by autonomous driving software.

I'm not entirely convinced it's a good idea, but one convincing argument I've seen is that sensor fusion is hard and it's not a priori clear which sensor to believe when they provide conflicting information. It's entirely possible that they get better predictive accuracy from vision-only data then in the analogous situations with vision+radar data.

I don't find this argument convincing at all. Just ignore the lidar in situations where lidar and vision conflict and you have the original decision.

They probably started without lidar due to the cost (Tesla is one of few companies actually selling cars to customers, who are sensitive to price). The price of lidar has come down, but I would guess it requires a fair amount of expensive rearchitecture to add it in now, and they haven't bitten the bullet yet.

"Just ignore the lidar in situations where lidar and vision conflict and you have the original decision."

Then why have lidar? If it's only purpose is to agree with vision or you just discard it, then why even have it?

I’m arguing against the idea that conflicting sensors is an issue. Clearly you can do no worse with sensors A and B than with just sensor A.

You might want to not have sensor B to lower the cost (as I suggested). Or you might want to have both sensors A and B because the provide some synergies (as gkop pointed out, better range of view, increased confidence, etc.). But it’s definitely not enough to just argue that the sensors may conflict.

Hypothetically, 1. Improved confidence when both signals agree 2. Improved field and range of view

Elon has said many times they would use Lidar if they thought it was needed / beneficial. In fact he and the SpaceX team designed a Lidar into the Dragon spacecraft to help it dock with the ISS (which is obviously a HIGHLY sensitive operation with extremely strict rules), and he said it was perfect for that scenario.

He was asked point blank if they would use Lidar as Tesla even if they were free... and he said no. In their opinion it's simply not the right approach to generate photons and read them as they come back. It's preferable to simply read the photons that are already coming in.

Sounds like a good reason to dump LIDAR. if vision is good enough to always defer to, why bother with something contradicting?

See my other comment. I’m making a counter argument to the specific claim that conflicting sensors is an issue. I have no idea if the added benefit of Lidar outweighs the costs, but it doesn’t seem logical that “conflicting signals” could be an issue.

The industry has been sensor fusion for decades. Training Autopilot neural networks to hallucinate road parts is easy but doing uncertainty estimation on simple radar measurements is hard? Also you still have to fuse the data from the multiple cameras.

I'm almost doubting my entire knowledge in the field now when all of a sudden everyone just accepts that "sensor fusion is hard" statement.

I'd imagine sensor fusion in this context isn't a singular problem right? It seems entirely plausible that for Tesla's system, it's straightforward to integrate cameras where the resolution is known and data is homogenous in a certain way, and that it's hard to integrate and use radar data into that system.

The Model 3 has a ARS-4B radar and these things usually already do perception for you, so the output is a list of vehicle velocity and pose. Given what I know about Autopilot , I don't see how is that hard to fuse. And I doubt the supplier of the radar (Continental) gave Tesla raw data access.

I just think there's more to the story than we're getting. Impossible to tell for an outsider though.

This is just my personal belief: but I really don't think an AI advanced enough to actually do self-driving will have trouble getting all the information it needs from cameras.

It seems to me that accurately determining signs, construction work, pedestrians, bikes, and even the intention/direction of these, is way harder than just reconstructing a 3D space from the cameras. I think the neural nets will have to do this internally anyway to be able to predict the direction a pedestrian is about to walk.

So I think LIDAR may be a benefit for limited self-driving in the short term. If it's a benefit in the long term remains to be seen, but there's good arguments that Teslas gamble will pay off, as they're allowed to use all their focus on the part of the AI that's most important for daily driving.

like 90%+ of living beings on the planet use eyes to navigate. Seems to work for them. None use LIDAR or RADAR, only a few use SONAR.

By that reasoning, shouldn’t the fastest living beings move around on wheels instead of legs?

Just because something works for evolution/nature, it doesn’t mean that there isn’t a simpler/better engineering alternative.

We have spend hopelessly many years trying to build machines that walk — Now Boston Dynamics seem to be there. But it is really complicated compared to wheels, and their machines are not running at 2-300 km/h.

Birds do exist and can reach speeds up to 200 mph (320 km/h).

Someone disputes that birds can fly fast and do visual navigation?

Computers sometimes need additional aid to compensate for the lack of human brain.

Possibly patent constrained?

> If you have access to LIDAR and RADAR, why not use them?

I think LIDAR is a very expensive part (~$10,000). Also, it is difficult to integrate with the car's design such that it does not make the car ugly. Tesla probably does not want to do a separate self-driving testing fleet, instead they want to use the existing Teslas on road to get training data. So they cannot use LIDAR in all cars sold to users.

I am sure if fully functional LIDAR units magically start selling for $100-$500 Tesla will start using them.

Unnecessary cost and complexity are the reasons Tesla has quoted before. One less sensor to buy, or replace when it breaks.

I’m not convinced. There’s good reasons they’re the lone player in the self driving space with this vision only play.

> When radar and vision disagree, which one do you believe? Vision has much more precision, so better to double down on vision than do sensor fusion.

[1] https://twitter.com/elonmusk/status/1380796939151704071

As I’ve commented above, “conflicting sensors” doesn’t make much sense as an argument. If sensors A and B conflicted, you could just ignore sensor B and get performance at least as good as just having sensor A.

Does sensor B have enough other benefits to outweigh the costs of adding it? I don’t know, but that’s the actual crux of this problem.

Sensor B “enhancing” sensor A is the exact same situation as a sensor conflict. How do you tell them apart? You’re oversimplifying the issue as if it was a matter of simple addition.

It's a shame people at Boeing seem to have the same outlook on this question. Cost a few hundred people their lives recently. When two sensors disagree, you don't know which one is right, but you know there's something wrong. You're going to fail, but at least you can fail safe. With only one sensor, you don't have that option.

Hence my question, "How much precision do you need to keep from running into a parked truck?"

I don’t see how “enhancing” and “conflict” are the same. As gkop pointed out, the two sensor types may have different ranges of view, or may agree about situations where just sensor A may be uncertain.

How much precision do you need to keep from running into a truck?

Since Lidar doesn't use moving parts I imagine it should be easy to make it last longer than the life of the vehicle.

You are confusing radar and lidar I think. True solid state automotive lidar is largely still in a research state. Most 'solid state' lidar sensors actually have very tiny moving parts in the form of MEMS assemblies. Even more delicate than a macro mechanical assembly.

But for radar, correct, those modules are available as a commodity for automotive lifescales.

The general problem is that simple, cost effective radar is low precision while expensive, complex lidar is high precision.

Looks like the 2022 Wey SUV from Great Wall Motors will have a true solid state lidar from Ibeo / ZF.

Because cars are currently being sold as FSD ready so you can't ever release LIDAR/RADAR after the fact.

Current Teslas are already fitted with radar, though Tesla wants to remove it from future models.