> So it’s not a surprise to see the low end models with lidar.
They could be going for a Tesla-esque approach, in that by equipping every car in the fleet with lidar, they maximise the data captured to help train their models.
And if he still doesn’t realize and admit he is wrong then he is just plain dumb.
Pride is standing in the way of first principles.
And he’s not wrong that roads and driving laws are all built around human visual processing.
The recent example of a power outage in SF where lidar powered Waymo’s all stopped working when the traffic lights were out and Tesla self driving continued operating normally makes a good case for the approach.
So to clarify, it wasn’t entirely a lidar problem it was an need to call home to navigate.
And people die all the time.
> The recent example of a power outage in SF where lidar powered Waymo’s all stopped working when the traffic lights were out and Tesla self driving continued operating normally makes a good case for the approach.
Huh? Waymo is responsible for injury, so all their cars called home at the same time DOS themselves rather than kill someone.
Tesla makes no responsibility and does nothing.
I can’t see the logic the brings vision only as having anything to do lights out. At all.
Yes... but people can only focus on one thing at a time. We don't have 360 vision. We have blind spots! We don't even know the exact speed of our car without looking away from the road momentarily! Vision based cars obviously don't have these issues. Just because some cars are 100% vision doesn't mean that it has to share all of the faults we have when driving.
That's not me in favour of one vs the other. I'm ambivalent and don't actually care. They can clearly both work.
Most of them cannot drive a car. People have crashes for so many reasons.
They do, but the rate is extremely low compared to the volume of drivers.
In 2024 in the US there were about 240 million licensed drivers and an estimated 39,345 fatalities, which is 0.016% of licensed drivers. Every single fatality is awful but the inverse of that number means that 99.984% of drivers were relatively safe in 2024.
Tesla provided statistics on the improvements from their safety features compared to the active population (https://www.tesla.com/fsd/safety) and the numbers are pretty dramatic.
Miles driven before a major collision
699,000 - US Average
972,000 - Tesla average (no safety features enabled)
2.3 million - Tesla (active safety features, manually driven)
5.1 million - Tesla FSD (supervised)
It's taking something that's already relatively safe and making it approximately 5-7 times safer using visual processing alone.
Maybe lidar can make it even better, but there's every reason to tout the success of what's in place so far.
Comparing the subsets of driving on only the roads where FSD is available, active, and has not or did not turn itself off because of weather, road, traffic or any other conditions" versus "all drivers, all vehicles, all roads, all weather, all traffic, all conditions?
Or the accident stats that don't count an accident any collision without airbag deployment, regardless of injuries? Including accidents that were sufficiently serious that airbags could not or were unable to deploy?
I have no doubt that there are ways to take issue with the stats. I'm sure we could look at accidents from 11pm - 6am compared to the volume of drivers on the road as well.
In aggregate, the stats are the stats though.
Sure, and using lidar means you can use it anywhere a person can go in any other technology too.
Tesla specifically decided not to use the taxi-first approach, which does make sense since they want to sell cars. One of the first major failures of their approach was to start selling pre-orders for self driving. If they hadn’t, they would not have needed to promise it would work everywhere, and could have pivoted to single city taxi services like the other companies, or added lidar.
But certainly it all came from Musk’s hubris, first to set out to solve the self driving in all conditions using only vision, and then to start selling it before it was done, making it difficult to change paths once so much had been promised.
The absolute genius made sure that he can't back out without making it bleedingly obvious that old cars can never be upgraded for a LIDAR-based stack. Right now he's avoiding a company-killing class action suit by stalling, hoping people will get rid of HW3 cars, (and you can add HW4 cars soon too) and pretending that those cars will be updated, but if you also need to have LIDAR sensors, you're massively screwed.
Same deal with his comments about how all anti-air military capability will be dominated by optical sensors.
This had happened a load of times with him. It seemed to ramp up around paedo sub, and I wonder what went on with him at that time.
History is replete with smart people making bad decisions. Someone can be exceptionally smart (in some domains) and have made a bad decision.
> He seemed to have drank his own koolaid back then.
Indeed; but he was on a run of success, based on repeatedly succeeding deliberately against established expertise, so I imagine that Koolaid was pretty compelling.
I hate Elon's personality and political activity as much as anyone, but it is clear from technical PoV that he did logical things. Actually, the fact that he was mistaken and still managed to not bankrupt Tesla is saying something about his skills.
Robot arms are neither a low-volume unique/high-cost market (SpaceX), nor a high-volume/high-margin business (Tesla). On top of that it's already a quite crowded space.
With vision you rely on external source or flood light. Its also how our civilization is designed to function in first place.
Anyway, the whole self driving obsession is ridiculous because being driven around in a bad traffic isn’t that much better than driving in bad traffic. It’s cool but can’t beat a the public infrastructure since you can’t make the car dissipated when not in use.
IMHO, connectivity to simulate public transport can be the real sweet spot, regardless of sensor types. Coordinated cars can solve traffic and pretend to be trains.
Agreed that public transportation is usually the best option in either case, though.
Right now I don't even have a car but for getting around outside of the city it's difficult sometimes.
Avoiding potholes is the hardest part of driving, really.
It's not that it's hard but I just hate it.
All reasons why I think public transit is the better solution over self driving cars. They're generally much safer, and also you get to do something while you're on the go. Pretty nifty, I think.
I love public transport and an added benefit is: I don't have to go back to where I left it. I often take a metro from A to B, walk to C and then get a bus back to A or something. Can't do that with a car, as such I tend to walk a lot more now. Because it's a hassle-free option now. The world seems more open for exploration when I don't have to worry about returning to the car, or having a drink, or the parking meter expiring. I really don't get that people consider cars freedom.
Of course once you go outside the city it's a different story, even here in Europe. Luckily I don't need to go there so much. But that's something that should be improved. On the weekend here in the city the metro runs 24/7 and the regional trains really should too but they don't.
you know a lot about the light you are sending, and what the speed of light is, so you can filter out unexpected timings, and understand multiple returns
Coordinated cars won't work unless all cars are built the same and all maintained 100% the same and regularly inspected. You can't have a car driving 2 inches from the car in front, if it can't stop just as fast as the car in front. People already neglect their cars, change brake compounds, and get stuck purchasing low quality brake parts due to lack of availability of good components.
Next time you see some total beater driving down the road, imagine that car 2 inches off your rear bumper, not even a computer can make up for poor maintenance. Imagine that 8000lb pickup with it's cheap oversized tires right in your rearview mirror with it's headlights in your face. It's not going to be able to stop either.
The good news is they're all commodity hardware prices now.
Tesla removing radar and parking ultrasonic sensors was a self own. Computer vision inference is pretty bad when all the camera sees is a while wall when backing up.
Fog - Radar will perceive the car. Multi car crash, long range radar picks it up.
Bright glare from sun, lidar picks it up. Lidar misses something, camera picks it up.
Waymo has the correct approach on perception. Jam with sensor so they have superhuman vision of environment around.
Whoever was in control. This isn’t some weird legal quagmire anymore, these cars are on the road.
And will continue to be until every municipality implements laws about it.
It’s not a conundrum as much as an implementation detail. We’ve decided to hold Waymo accountable. We’re just ticking the boxes around doing that (none of which involve confusion around Waymo being responsible).
The nature of the punishment does not necessarily follow the same rules as for human incompetence, e.g. if the error occurs due to some surprising combination of circumstances that no reasonable tester would have thought to test, which I can't really give an example of because anything I can think of is absolutely something a reasonable tester would have thought to test, but for the sake of talking about it without taking this too seriously consider if a celebrity is crossing a road while a large poster of their own face is right behind them.
Don't get me wrong, perfection should be the long term goal. However I will settle for less than perfection today so long as it is better.
Though better is itself hard to figure out - drunk (or otherwise impaired drivers) are a significant factor in car deaths, as is bad weather when self driving currently doesn't operate at all. Statistics do need to make sure self driving cars are better than non-impaired drivers in all situations where humans driver before they can claim better. (I know some data is collected, but so far I haven't seen any independent analysis. The potentially biased analysis looks good though - but again it is missing all weather conditions)
The benefits of self-driving should be inrefutable before requiring it. At least x10 better than human drivers.
Right now… Tesla likes to show off stats that suggest accidents go down while their software is active, but then we see videos like this, and go "no sane human would ever do this", and it does not make people feel comfortable with the tech: https://electrek.co/2025/05/23/tesla-full-self-driving-veers...
Every single way the human vision system fails, if an AI also makes that mistake, it won't get blamed for it. If it solves every single one of those perception errors we're vulnerable to (what colour is that dress, is that a duck or a rabbit, is that an old woman close up facing us or a young woman from a distance looking away from us, etc.) but also brings in a few new failure modes we don't have, it won't get trusted.
If you charged car makers $20m per pedestrian killed by their cars regardless of fault you'd probably see much safer designs.
This is an extremely optimistic view on how companies work
A big reason car companies don't worry much about killing pedestrians at the moment is it costs them ~$0.
About half our road fatalities are pedestrians. About 80% of those are intoxicated with alcohol. When you're driving at 40mph, at night, and some drunk guy chooses to cross the road, no amount of safety features or liabilities can save him.
Sure, cars can be safer for light collisions with pedestrians where the car is going slowly. Especially in the US where half the cars have a very high hood. But where I live the problem is not safer cars, it's drunk pedestrians.
Here in the UK we have a standard 30mph built up area limit, dropping to 20mph in most residential area.
Result - a massive reduction in serious injuries and fatalities, especially in car - pedestrians collisions.
And sometimes at places that aren’t even a cross walk.
Im in DTLA frequently and I am almost even developing a secondary instinct to cover my brake and have an extra look around when a Waymo stops in a street.
Because it may be dropping off or picking up a rider or it saw something or someone I didn’t. Just happened Saturday in fact. I saw it do an abrupt stop when I was yielding to it at a “T” intersection and expected it to have the right of way and keep going. I didn’t proceed until I could figure out WHY it had just stopped, like “okay WHERE’S the passenger”
and then five or so people started running across the street in front of it that I would not have seen if that Waymo wasn’t there and I was clear to turn left.
As an added bonus it stayed stopped after they all crossed and I decided to be a jerk and turn left in front of it. It stayed stopped for me too. There’s no driver in it. It ain’t mad. XD
I have a good eye for spotting uber drivers who are about to load or unload too, Especially if they have some common sense and are trying to line up to do that so their passenger can get on or off curbside. A Waymo is just.. way more immediately identifiable that I can react that much faster to it or just be like.. alright. I’ll take a cue from it, it’s usually right.
And hell even if it’s wrong, maybe this isn’t a good time to pull out in front of it anyway!
This is basically what we have (for reasonable definitions of full).
If there are single bulbs displaying red, green and yellow please give clear examples.
Potentially as extraneous as range to a surface that a camera can’t tell apart from background.
More to the point, everyone but Tesla is doing cameras plus Lidar. It’s increasingly looking like the correct bet.
At what proportion? Is it mostly lidar or mostly cameras? Or 50/50?
> Potentially as extraneous as range to a surface that a camera can’t tell apart from background.
I guess yeah for backside of the car you'd probably better off measuring actual actions.
How about when you come 4 way stop. LIDAR is useless as it wouldn't recognize anyones turn signals.
What proportion of your vision is rods or cones? Depends on the context. You can do without one. But it’s better with both.
> How about when you come 4 way stop. LIDAR is useless as it wouldn't recognize anyones turn signals
Bad example. 99% of a 4-way stop is remembering who moved last, who moves next by custom and who may jump the line. What someone is indicating is, depending on where you are, between mildly helpful and useless.
eg A driving decision system needs to know object distances AND traffic light colours. It doesn't particularly need to know the source of either. You could have a camera-only system that accurately determines colour and fuzzy-determines distance. Or you could have a LIDAR-only system that accurately determines distance and fuzzy-determines colour.
Or you use both, get accurate LIDAR-distance and accurate camera-colour and skip all the fuzzy-determination steps. Or keep the fuzzy stuff and build a layer of measurement-agreement for redundancy.
So then the question becomes, what's your proportion when deciding whether to stop at a traffic light? Is it mostly light colour or mostly distance to other objects? Or 50/50?
I'd say it's 100/100.
Like them, I don't understand what you're asking by "proportion". Bits/second? Sensor modules/vehicle? Features detected by the AI?
Because it really feels like you're just JAQing off, either to come to some "gotcha" moment or a pseudo-intellectual exercise. In either case, the questions feel bad faith.
-- but I'm not sure how to get data on ex. how much Tesla is charged for a Nvidia whatever or what compute Waymo has --
My personal take is Waymo uses cameras too so maybe we have to assume the worst case, +full cost of lidar / +$130
The problem with Tesla is, that they need to combine the outputs of those camera's into a 3d view, what takes a LOT more processing power to judge distances. As in needing more heavy models > more GPU power, more memory needed etc. And still has issues like a low handing sun + white truck = lets ram into that because we do not see it.
And the more edge cases you try to filter out with cameras only setups, the more your GPU power needs increase! As a programmer, you can make something darn efficient but its those edge cases that can really hurt your programs efficiency. And its not uncommon to get 5 to 10x performance drops, ... Now imagine that with LLM image recognition models.
Tesla's camera only approach works great ... under ideal situations. The issue is those edge cases and not ideal situations. Lidar deals with a ton of edge cases and removes a lot of the progressing needed for ideal situations.
>I'd love to take on this challenge: the article they linked shows the cost add for LIDAR (+$130)
The article claims that, but when you actually try to follow the source it fails the fact check.Even ignoring various current issues with Lidar systems that aren’t fundamental limitations, large amounts of road infrastructure is just designed around vision and will continue to be for at least another few decades. Lidar just fundamentally can’t read signs, traffic lights or road markings in a reliable way.
Personally I don’t buy the argument that it has to be one or the other as Tesla have claimed, but between the two, vision is the only one that captures all the data sufficient to drive a car.
> Lidar just fundamentally can’t read signs, traffic lights or road markings in a reliable way.
Actually, given that basically every meaningful LIDAR on the market gives an "intensity" value for each return, in surprisingly many cases you could get this kind of imaging behavior from LIDAR so long as the point density is sufficient for the features you wish to capture (and point density, particularly in terms of points/sec/$, continues to improve at a pretty good rate). A lot of the features that go into making road signage visible to drivers (e.g. reflective lettering on signs, cats eye reflectors, etc) also result in good contrast in LIDAR intensity values.
It's like having 2 pilots instead of 1 pilot. If one pilot is unexpectedly defective (has a heart attack mid-flight), you still have the other pilot. Some errors between the 2 pilots aren't uncorrelated of course, but many of them are. So the chance of an at-fault crash goes from p and approaches p^2 in the best case. That's an unintuitively large improvement. Many laypeople's gut instinct would be more like p -> p/2 improvement from having 2 pilots (or 2 data streams in the case of camera+LIDAR).
In the camera+LIDAR case, you conceptually require AND(x.ok for all x) before you accelerate. If only one of those systems says there's a white truck in front of you, then you hit the brakes, instead of requiring both of them to flag it. False negatives are what you're trying to avoid because the confusion matrix shouldn't be equally weighted given the additional downside of a catastrophic crash. That's where two somewhat independent data streams becomes so powerful at reducing crashes, you really benefit from those ~uncorrelated errors.
This is not going to be true for a very long time, at least so long as one's definition of "vision" is something like "low-cost passive planar high-resolution imaging sensors sensitive to the visual and IR spectrum" (I include "low-cost" on the basis that while SWIR, MWIR, and LWIR sensors do provide useful capabilities for self-driving applications, they are often equally expensive, if not much more so, than LIDARs). Camera sensors have gotten quite good, but they are still fundamentally much less capable than the human eyes plus visual cortex in terms of useful dynamic range, motion sensitivity, and depth cues - and human eyes regularly encounter driving conditions which interfere or prohibit safe driving (e.g. mist/ fog, heavy rain/snow, blowing sand/dust, low-angle sunlight at sunrise/sunset/winter). One of the best features of LIDAR is that it is either immune or much less sensitive to these phenomena at the ranges we care about for driving.
Of course, LIDAR is not without its own failings, and the ideal system really is one that combines cameras, LIDARs, and RADARs. The problem there is that building automotive RADAR with sufficient spatial resolution to reliably discriminate between stationary obstacles (e.g. a car stalled ahead) and nearby clutter (e.g. a bridge above the road) is something of an unsolved problem.
By the way, Tesla engineers secretly trained their vision systems using LIDAR data because that's how you get training data. When Elon Musk found out, he fired them.
Finally, your premise is nonsensical. Using end to end learning for self driving sounds batshit crazy to me. Traffic rules are very rigid and differ depending on the location. Tesla's self driving solution gets you ticketed for traffic violations in China. Machine learning is generally used to "parse" the sensor output into a machine representation and then classical algorithms do most of the work.
The rationale for being against LIDAR seems to be "Elon Musk said LIDAR is bad" and is not based on any deficiency in LIDAR technology.
Imagine if the watch simply tells you if it is safe to jump into the pool (depending on the time it may or may not have water). If watches conflict, you still win by not jumping.
So in a way, yes.
The main difference is that "don't know the time" is a trivial consequence, but "crash into a white truck at 70mph" is non-trivial.
But it's the same statistical reasoning.
I know there are theoretical and semi-practical ways of reading those indicators with features that are correlated with the visual data, for example thermoplastic line markings create a small bump that sufficiently advanced lidar can detect. However, while I'm not a lidar expert, I don't believe using a completely different physical mechanism to read that data will be reliable. It will surely inevitably lead to situations where a human detects something that a lidar doesn't, and vice versa, just due to fundamental differences in how the two mechanisms work.
For example, you could imagine a situation where the white lane divider thermoplastic markings on a road has been masked over with black paint and new lane markings have been painted on - but lidar will still detect the bump as a stronger signal than the new paint markings.
Ideally while humans and self driving coexist on the same roads, we need to do our best to keep the behaviour of the sensors to be as close to how a human would interpret the conditions. Where human driving is no longer a concern, lidar could potentially be a better option for the primary sensor.
Conflicting lane marking due to road work/changes is already a major problem for visual sensors and human drivers, and something that fairly regularly confuses ADAS implementations. Any useful self-driving system will already have to consider the totality of the situation (apparent lane markings, road geometry, other cars, etc) to decide what "lane" to follow. Arguably a "geometry-first" approach with LIDAR-only would be more robust to this sort of visual confusion.
The focus shouldn't be on which sensor to use. If you are going to use humans as examples, just take the time to think how a human drives. We can drive with one eye. We can drive with a screen instead of a windshield. We can drive with a wiremesh representation of the world. We also use audio signals quite a bit when when driving as well.
The way to build a self driving suite is start with the software that builds your representation of the world first. Then any sensor you add in is a fairly trivial problem of sensor fusion + Kalman filtering. That way, as certain tech gets cheaper or better or more expensive and worse, you can just easily swap in what you need to achieve x degree of accuracy.
We truly have no understanding of how the human brain really models the world around us and reasons over motion, and frankly anyone claiming to is lying and trying to sell something. "But humans can do X with just Y and Z..." is a very seductive idea, but the reality is "humans can do X with just Y, Z, and an extremely complex and almost entirely unknown brain" and thus trying to do X with just Y and Z is basically a fool's errand.
> ...builds your representation of the world first...
So far, I would say that one of the very few representations that can be meaningfully decoupled from the sensors in use is world geometry, and even that is a very weak decoupling because the ways you performantly represent geometry are deeply coupled with the capabilities of your sensors (e.g. LIDAR gives you relatively sparse points with limited spatial consistency, cameras give you dense points with higher spatial consistency, RADAR gives you very sparse targets with velocity). Beyond that, the capabilities of your sensors really define how you represent the world.
The alternative is that you do not "represent" the world but instead have that representation emerge implicitly inside some huge neural net model. But those models and their training end up even more tightly coupled to the type of data and capabilities of your sensors and are basically impossible to move to new sensor types without significant retraining.
> Then any sensor you add in is a fairly trivial problem of sensor fusion + Kalman filtering
"Sensor fusion" means everything and nothing; there are subjects where "sensor fusion" is practically solved (e.g. IMU/AHRS/INS accelerometer+gyro+magnetometer fusion is basically accepted as solved with EKF) and there are other areas where every "fusion" of multiple sensors is entirely bespoke.
But if you use "roomba" as a generic term for robot vacuum then yes, Chinese Ecovacs and Xiaomi introduced lidar-based robot vacuums in 2015 [2].
[1] https://www.theverge.com/news/627751/irobot-launches-eight-n...
[2] https://english.cw.com.tw/article/article.action?id=4542
My ex got a Roomba in the early 2010s and it gave me an irrational but everlasting disdain for the company.
They kept mentioning their "proprietary algorithm" like it was some amazing futuristic thing but watching that thing just bump into something and turn, bump into something else and turn, bump into something again and turn again, etc ... it made me hate that thing.
Now when my dog can't find her ball and starts senselessly roaming in all the wrong directions in a panic, I call it Roomba mode.
My neighbour used to park like that; "thats what the bumpers are for - bumping"
I think fsd should be both at minimum though. No reason to skimp on a niw inexpensive sensor that sees things vision alone doesn’t.
Yes, humans don’t have built in lidar. But humans do use tools to augment their capabilities. The car itself is one example. Birds don’t have jet engines, props, or rotors… should we not use those?
I mean, you have to have vision to drive. What are you getting at? You can't have a lidar only autonomous vehicle.
>Lidars come down in price ~40x.
Is that really true? Extraordinary claims require extraordinary proof.Ars cites this China Daily article[0], which gives no specifics and simply states:
>A LiDAR unit, for instance, used to cost 30,000 yuan (about $4,100), but now it costs only around 1,000 yuan (about $138) — a dramatic decrease, said Li.
How good are these $138 LiDARs? Who knows, because this article gives no information.This article[1] from around the same time gives more specifics, listing under "1000 yuan LiDARs" the RoboSense MX, Hesai Technology ATX, Zvision Technologies ZVISION EZ5, and the VanJee Technology WLR-760.
The RoboSense MX is selling for $2,000-3,000, so it's not exactly $138. It was going to be added to XPENG cars, before they switched away from LiDAR. Yikes.
The ATX is $1400, the EZ5 isn't available, and the WLR-760 is $3500. So the press release claims of sub-$200 never really materialized.
Furthermore, all of these are low beam count LiDARs with a limited FOV. These are 120°x20°, whereas Waymo sensors cover 360°x95° (and it still needs 4 of them).
It seems my initial skepticism was well placed.
>if you had to choose one or the other for cost you’d be insane to choose vision over lidar
Good luck with that. LiDAR can't read signs.[0] https://global.chinadaily.com.cn/a/202503/06/WS67c92b5ca310c...
[1] https://finance.yahoo.com/news/china-beijing-international-a...
The argument is that humans provide a proof-of-concept that vision + neural net can drive a car, because (for some reason) some people doubt this is possible. However there's no need for a "proof of concept" that mechanical legs can be built, because everyone already know that building mechanical legs is possible.
Waymo has certainly had its share of issues lately on the "practicality" axis. The cost of (actually good enough) LiDAR hardware doesn't help practicality either.
They surely have as Waymo have a much better safety record than Tesla and that's even with Tesla hiding and obscuring their crash data - probably outright lying about most crashes if Musk has any input to their processes.
The big advantage of Lidar is that it can easily pick out obstructions and that's a good ability for a driver to have, whereas Teslas can get fooled by shadows etc. It seems obvious to me that having cameras and Lidar allows for much better analysis of traffic and pedestrians.
>They surely have as Waymo have a much better safety record than Tesla
Tesla is already closing in on Waymo to within a factor of 2[0], and this is less than 6 months in, with Waymo having 125 million autonomous miles and Tesla having under a million. Bet on whichever horse you want, but this trend is not looking good for Waymo. >It seems obvious to me
Oh thank goodness! Fortunately this style of argument has never been wrong before. ;)[0] https://mashable.com/article/tesla-robotaxis-with-human-safe...
LIDAR is also straight up worthless without an unholy machine learning pipeline to massage its raw data into actual decisions.
Self-driving is an AI problem, not a sensor problem - you aren't getting away from AI no matter what you do.
https://cleantechnica.com/2025/03/20/lidars-wicked-cost-drop...
Meanwhile visible light based tech is going up in price due to competing with ai on the extra gpu need while lidar gets the range/depth side of things for free.
Ideally cars use both but if you had to choose one or the other for cost you’d be insane to choose vision over lidar. Musk made an ill timed decision to go vision only.
So it’s not a surprise to see the low end models with lidar.