Level 5 self driving is still theoretical at this point, and theoretically speaking, there is no reason to assume LIDAR is absolutely necessary for it. Computer vision based pipelines are theoretically capable of providing the same depth information as LIDAR, using stereo, camera arrays or even monocular SLAM.

The main obstacle for self-driving car is much more on the processing side than on the sensor side. After all, humans are able to drive with audio + stereo vision that's not even 360 degrees.

It could very well be that the reason Tesla's statement about "full self-driving car HW" is inaccurate is because of the lack of sufficient onboard computing power rather than the lack of sensors (IIRC they have an Nvidia X2 and GP106 onboard, which only have a total of about 4 SPFP TFLOPs).

No matter which way you slice it, Tesla should not be claiming the hardware required is present if the hardware requirements are, at best, theoretical.

If it turns out the hardware is not good enough I would expect them to replace it with something that is once hardware requirements are known.

They've given themselves enough wiggle room to never roll out Level 5 automation (and thus never admit they were wrong about what hardware it would require.)

>"Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction. It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval."

True. Tesla also claims to not really advertise.

Well, spinning hyperbole that fires up the clickbait is definitely effective advertising.

No, advertising is paid placement. What you’re describing is PR.

Tesla will in some form, though it’s hard to guess what color the money will be from the outside. It would be a goodwill jesture, so probably out of a marketing, customer retention or warrenty account.

If they upgrade hardware more than once I would worry about where the money came from.

>"Computer vision based pipelines are theoretically capable of providing the same depth information as LIDAR, using stereo, camera arrays or even monocular SLAM."

Are there any companies that are taking the purely computer vision approach then?

>"(IIRC they have an Nvidia X2 and GP106 onboard, which only have a total of about 4 SPFP TFLOPs)."

Can you say how far off is that compute power from what is actually required for a "full self-driving" car?

Comma.ai

> every expert in the field unconnected to Tesla seems to agree [lidar] is necessary to implement Level 5 self driving cars

As someone that doesn't know anything about self-driving cars, how can this be true? Humans can drive cars with only visual input. Why can't AI?

Human eyes don't work anything like cameras. The eye processes information that isn't just intensities of photons and their wave lengths—which is the only information collected by cameras. See Walter Pitt's "What the Frog Eyes Tell the Frog Brain" for an introduction: 'Fundamentally it shows that the eye speaks to the brain in a language already highly organized and interpreted, instead of transmitting some more or less accurate copy of the distribution of light on the receptors.'

A simple abstraction here: we put two high-quality monitors before our eyes and get all the data from them. Can we continue driving this way? Absolutely. This means that no matter how our eyes actually work, an array of RGB pixels is enough. I'd go further and say that a 64 x 64 grayscale camera at 10 fps is enough to drive a car. The real magic happens in the brain that reconstructs the 3D model and predicts what happens next.

> "64 x 64 grayscale camera at 10 fps is enough to drive a car."

If you had vision that bad, the State of California would not let you get a drivers license, they require at least one of your eyes to be 20/40. Anyway, the point of the GP is that the brain isn't the only piece of wetware doing image processing, but more to the point the current state of the art CV really is not up to the task.

And it really doesn't seem like it's going to be there anytime soon.

That is an analogy not an abstraction but if you had read the paper or the single quote I provided, you would know that even relatively simple eyes (such as those of the frogs which cannot see still objects) don't only process a distribution of light intensity and frequency. The eye measures other information that is lost by the camera including photon phase, arrival time, polarization, orbital angular momentum, linear momentum, and probably many more measurements. More so, this is done by specialized organelles of the eye rather than by the brain.

It's great that eyes can detect polarization and orbital momentum. My point is that none of that info is needed to drive a car.

That's true, but the main point is that current software still relies on sampling. Eyes don't sample.

And this is irrelevant also, because we can introduce sampling the way I described and can continue driving just fine.

I've thought about it and yes this does make sense.

Humans can do a lot of things we don't understand how to convince software (...or other humans who don't intrinsically learn it very well) to do.

Lidar lets you get a high degree of accuracy for certain variables in your model - because you can know fairly accurately when and how strong the signal you sent out is, you can have a fairly high degree of confidence when you get reflection of how far away the object is, and feed that into your persistent idea of the world, versus having to use a bunch of heuristics to try and extract that from purely audiovisual data.

(That's my understanding, at least - I don't work in that industry.)

A good way IMO to think about it is by looking at airplanes. Birds fly by flapping their wings to generate thrust, but planes use turbines to generate thrust.

Just because humans use vision to drive, doesn't mean that we should restrict ourselves to vision for cars. Also, as another comment points out, our visual system does a lot more than just 'see' in the literal light/color processing sense that a camera does

Because current AI does not seem as good at generalizing as humans?

Since level 5 driving has not yet been achieved, aren't we talking about future AI, not current AI?

Tesla's marketing team wants people buying cars to believe their Tesla will support Level 5 automation sometime within the usable lifespan of that car. They're not bold enough to claim it's a near future inevitability, but they're leaving that implied possibility open to get consumers excited about it.

Google/Waymo has obviously been throwing a ton of resources at nailing computer vision, their recaptcha that's plainly training such a system is doubtlessly just the publicly visible tip of an iceberg. Yet they're still investing into LIDAR as well. If Google/Waymo is bearish on a pure CV solution, I don't see any justification for Tesla thinking they can do it. Not in any near future.

https://arstechnica.com/cars/2017/01/googles-waymo-invests-i...

Elon said it before, LIDER only gives you locally optimal solution, and it's not enough to build a full self-driving vehicle.

LIDAR doesn't compute solutions, it's a sensor that gives you accurate information about the environment. Far more precise and accurate than any current computer vision system. What you do with that data is up to the rest of the system.

Elon has publicly trashed LIDAR plenty of times. I think you should treat his rants as defenses of his products' reputation, not earnest impartial engineering appraisals. If Elon were to admit that LIDAR will play a vital role in Level 5 automation, he'd be admitting that his company has been mistaken or lying for years. I think Elon's personality precludes him coming clean on something like this.

> "it's not enough to build a full self-driving vehicle."

As far as I know, nobody is trying to build a Level 5 system using only LIDAR. The serious teams who are working on this problem are all using LIDAR in addition to cameras and radars. LIDAR can't read a street sign; cameras and computer vision will obviously play an important role in Level 5 systems for that reason and more. But the teams that aren't trying to use automation hype to sell luxury cars don't share Elon's enthusiasm for CV/radar with no LIDAR.

Isn't lidar super cheap now? Couldn't they retrofit it.

> their cars do not contain LIDAR, which essentially every expert in the field unconnected to Tesla seems to agree is necessary to implement Level 5 self driving cars

Calm down a second.

Elon/Tesla are trying to do things differently/do things that other people are not or think can't be done.

To name a few"

Every "export in the field" said you can't land an orbital class booster vertically.

Every "expert in the field" has not figured out how to save billions and years digging tunnels.

Every "expert in the field" seems to design electric cars that look like a cartoon joke.

Every "expert in the field" was pretty sure Tesla could never be profitable.

Don't confuse "has never been done before" with "Can't be done". Let the results speak for what's possible.

I think “every expert” may be stretching the truth, unless you are proclaiming that Tesla has no experts working on this.

They said

> every expert in the field unconnected to Tesla

The edit button is a wonderful tool.

Besides, the fact that I can drive my car relying primarily on two eyes and a brain suggests that it must be possible to extract and process the information to drive a car with stereo vision, and without LIDAR. LIDAR may become cheap and small enough that it should be added purely from a safety perspective - more better information is better. However, I think plenty of experts would agree that theoretically it is possible without.

Here are some experts that think it works without LIDAR, so you may wish to edit “every”. https://arxiv.org/pdf/1708.09839.pdf

Even if they can make it work, they note that Tesla's hardware is still inadequate. Tesla boast 8 cameras providing 360 degree coverage, but the authors of that paper are unimpressed:

>"The autopilot feature [19] in cars produced by Tesla Motors enables autonomous driving on highways through a combination of a forward-looking camera and radar. Due to the limited field-of-view, these cars cannot execute autonomous driving in cities which require surround perception."

So while you may have found some experts who think Level 5 automation can be done using only CV, you've only reinforced my main point that Elon Musk is not a trustworthy person.

> every expert in the field unconnected to Tesla

I'm saying that when you ask people not getting a paycheck from Tesla, they say that LIDAR is required. Very notably including Mobileye, who severed their connection with Tesla over this disagreement. Mobileye is particularly notable since they invested heavily into camera-only solutions; Mobileye has a strong financial incentive to dismiss LIDAR, yet they don't.

It's obvious why Tesla doesn't either put LIDAR in all their cars or remove the self driving hardware claims from their advertising; Elon wants to have his cake and eat it too.

More from mobileye:

> Level 3 – Autonomous Under Certain Circumstances [...] Mobileye expects this additional redundancy to be covered by additional sensors like radar and lidar (for shape and object detection) and by Mobileye’s REM™ localization map for identification of safe drivable paths and knowledge of intersections and other traffic signage or instructions.

https://www.mobileye.com/future-of-mobility/history-autonomo...

Harsher words than mine, from GM’s director of autonomous vehicle integration Scott Miller:

>“The level of technology and knowing what it takes to do the mission, to say you can be a full Level 5 with just cameras and radars is not physically possible,” said Miller. He went on to add that Musk is “full of crap.”

https://futurism.com/gm-expert-calls-elon-musk-full-of-crap-...

An article about the angular resolution limitations of radar (overcoming this is exactly what LIDAR is for):

>Raj Rajkumar, who researches autonomous driving at Carnegie Mellon University, thinks those assumptions concern one of Tesla's key sensors. “The radars they use are apparently meant for detecting moving objects (as typically used in adaptive cruise control systems), and seem to be not very good in detecting stationary objects," he says.

>That's not nearly as crazy as it may seem. Radar knows the speed of any object it sees, and is also simple, cheap, robust, and easy to build into a front bumper. But it also detects lots of things a car rolling down the highway needn't worry about, like overhead highway signs, loose hubcaps, or speed limit signs. So engineers make a choice, telling the car to ignore these things and keep its eyes on the other cars on the road: They program the system to focus on the stuff that's moving.

https://www.wired.co.uk/article/tesla-fire-truck-crash-autop...

If you search around for this, you'll find there are A LOT of people calling BS on Tesla's claims that cameras and radar are adequate sensors for Level 5 automation. Tesla's website hints at none of this. The only caveats I see from them are concerning software and regulation:

>Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction. It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval. Please note also that using a self-driving Tesla for car sharing and ride hailing for friends and family is fine, but doing so for revenue purposes will only be permissible on the Tesla Network, details of which will be released next year.

Required for what, the self driving cars that don't exist yet? Radar and computer vision is simply a separate approach, not a wrong one. Tesla is as much an expert in the field as anyone, given that, you know, they have the best self driving commercial product by far.

>"Required for what, the self driving cars that don't exist yet?"

Required for Level 5 automation. Tesla claims to be selling the required hardware, waiting only on the software and legislation allowing it. All other players in this field are focusing on a combination of cameras, radar and LIDAR.

Since LIDAR is expensive and the self-driving hype is real, Tesla have a strong short term financial incentive to forgo the former while claiming to sell the later. Combine that with virtually every other expert in the field saying they're full of BS, and I think the picture becomes crystal clear. Elon covers for himself by promoting conspiracy theories about grand industry-wide plots to keep an underdog down, and that has worked very well for him so far. But like most conspiracy theories, this is one you should think critically about.

I work for an automotive tech company and have installed an L2 self driving system into my friends Camry, I don't think you've thought critically about this. LIDAR is currently a prohibitively expensive technology that provides a point map of the surrounding area. The LIDAR that Waymo uses is nearly as expensive as the car itself. This same point map can be done with computer vision/cameras, and good software. There are many other "experts" that think this is equally viable e.g. George Hotz at Comma AI or Anthony Levandowski at Pronto.

I am not opposed to LIDAR. I just think it is unnecessary and that computer vision is likely to get good enough before LIDAR gets cheap enough.

The last time I looked into it (which was admittedly a few months ago) Comma.ai was by their own admission a Level 2 system, and a somewhat sketchy one at that. While it's obviously really cool, it's a long way off from Level 5.

LIDAR currently being very expensive (and a bit bulky) is obviously the reason Tesla hasn't put LIDAR in their cars. I don't really fault them for that, I fault them for their deceptive advertising. The LIDAR industry has not yet progressed to the point where it makes sense to bring a car to market with hardware ready for Level 5 automation, so Tesla should stop claiming they've done so.

Assuming the current Level 5 automation hype doesn't fizzle then smoulder for decades like the AI craze of the 60s, then it's success will coincide with rapid advances in size and price of LIDAR.

Cadillac’s super cruise usually gets higher ratings.

Go ahead and name a better self driving tech that you can buy right now. There is only 3 btw.

Tesla's Autopilot / GM's Supercruise (only available on Cadillac CTS and certain highways) / Comma AI's Openpilot.

You also misconstrued my point. I said best commercially available self driving. Seemingly Waymo has better tech but I can't buy or use a Waymo.

I'm curious, because I honestly dont know, how does LIDAR perform in suboptimal environments such as heavy fog , a Midwest downpour or a blizzard? So far, everything I've read has these cars testing in sunny day (no pun intended) scenarios, with the worst environmental threat being night-time.

Every system known to man, including man, struggles in those sort of conditions.

But this is an important factor to consider when evaluating safety statistics. Humans operate cars in all sort of adverse conditions and human accident rates include accidents in those kind of conditions. Accident rates for autonomous systems that refuse to attempt operation under anything less than ideal conditions won't be directly comparable to human accident rates... but expect this to be glossed over by automotive marketing departments who will eagerly compare apples to oranges as soon as the opportunity presents itself.

This is basically my concern with the hype over automated driving. I live in the upper Midwest. 6 months or more of the year have unideal driving conditions. A purely autonomous vehicle is worthless to me if it can't get me to where I need to go half the time.