He really embodies the ethos of "move fast and break things". So let's fire 80% of the staff, see what falls down, and rehire where we made "mistakes". I really think he has an alarmingly high threshold for the number of lives we can lose if it accelerates the pace of progress.
Which seems like an antidote to the current culture of "don't build anything anywhere."
"Not a single other automobile manufacturer or ADAS self-driving technology provider reported a single motorcycle fatality in the same time frame."
That's true, but it would obviously be completely unreasonable to stop all flying in the meantime.
As for "best technology available," the galaxy brains in this thread are tossing out numbers assuming that sensor fusion never fails and that correlated failure modes can be neglected, which is wild. I never thought Elon was a genius -- he's a business guy willing to make big bets on interesting tech, nothing more nothing less -- but if the confidently incorrect engineering claims on display in this thread are any indication, maybe you guys should be calling him a genius after all, because this ain't it.
Tesla releases all self driving statistics including time since disengagement of self-driving before accident, right?
The numbers include all incidents where self-driving was active within 30 seconds of the crash, because of course they do. The meme that Tesla is allowed to simply bypass reporting on a technicality would be absurd if it weren't so voraciously spread by post-truth luddites. Look, if you want to dunk on Elon, I get it, but he does so many real shitty things that I would ask you to focus on those and not make shit up.
https://www.reuters.com/business/autos-transportation/trump-...
A upper bound could be easily produced by Tesla. All vehicle fatalities identified by police are recorded in the national Fatality Analysis Reporting System (FARS) [1]. Tesla could investigate every recorded Tesla fatality by VIN and cross-reference with their telemetry to determine if their systems were available at all or active on or near the time of the fatal crash. This would cost them between a few thousand dollars to a few million dollars annually, depending on desired precision, which compared to the billions it makes annually is minuscule sum to affirmatively establish safety.
They intentionally choose to not do so. Instead deliberately deceiving consumers and the public by conflating the lower bound with a upper bound in all official safety messaging.
Even ignoring their gross incompetence or maliciousness in not doing simple safety analysis when numerous lives are on the line. The mere fact they conflate a lower bound with a upper bound in their messaging is scientific malfeasance of the highest order. Their reporting has zero credibility when they make such clear and intentional misreporting purely for their own benefit to the detriment of the public and even their own customers.
[1] https://www.nhtsa.gov/research-data/fatality-analysis-report...
Of course, at first this just substitutes one type of problem for another because even shitty drivers have to drive many lifetimes of distance before killing anyone so you wind up with the unfortunately named "shot noise" from small-N. But Tesla FSD drives so many miles that the statistics are no longer small-N. If supervised FSD was worse than human, we should be seeing many more bodies than we do, so supervised FSD is clearly better than human.
Of course, that number needs to be adjusted for critical interventions before we can say anything about unsupervised FSD, and in an ideal universe I wish Tesla was forced to disclose those figures. Unfortunately, we live in a universe where luddites are firmly in control of this conversation and I cannot deny that a forced disclosure would be heavily abused in a way that costs lives. Still, regulators get to see the numbers before approving unsupervised rollouts, and as a compromise this makes neither myself nor the luddites happy but I suppose it will do.
To your point about luddites, that’s why I think it’s erroneous to use “as good as a human” as the metric. It will need to be much better than a human but those people reluctantly trust it enough to hand over control.
What I literally said was Tesla doesn't release all of the self-driving statistics, like time to disengagement. They are noticeably more cagey about releasing that kind of info claiming it as "trade secret". Did you see the data sets you pointed to before pointing to them? Full of Tesla crashes, and full of redactions.
I don't need to dunk on Elon. He dunks on himself. Some people just worship him too hard to realize it.
While it was absolutely vital to getting the costs of the original Tesla Roadster and SpaceX launches way down… it can only work when you are able to accept "no, stop" as an answer.
Rockets explode when you get them wrong, you can't miss it.
Cars crashing more than other models? That's statistics, which can be massaged.
Government work? There's always someone complaining no matter what you do, very easy to convince yourself that all criticism is unimportant, no matter how bad it gets. (And it gets much worse than the worst we've actually seen from DOGE and Trump — I don't actually think they'll get to be as bad as the Irish Potato Famine, but that is an example of leaders refusing to accept what was going on).
Now the errors are not all independent so it's not as good as that, but many classes of errors are independent (e.g. two pilots getting a heart attack versus just one) so you can get pretty close to that p^n. Musk did not understand this. He's just not as smart as he's made out to be.
You have a main and a backup pilot, but either one must be 100% capable of doing it on their own. The backup is silently double checking, but their assignments are more about ensuring that the copilot doesn't just check out because they're human. If the copilot ever has to say "don't do that it's going to kill us all" it's a crisis.
Lidar is a good backup but the car must be able to with without it. You can't drive just with lidar; it's like driving by Braille. Lidar can't even read a stop light. If they cannot handle it with just the visuals, the car should be not be allowed on the road.
I concur that is terrifying that he was allowed to go without the backup that stops it from killing people. Human co drivers are not a good enough backup.
But he's also not wrong that the visual system must be practically perfect -- if it's possible at all. Which it surely isn't yet.
Trusting vision systems for 30 seconds or even 30 minutes over the lifetime of a car is very different than trusting them for 30,000 hours. So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them.
As a comparison for planes, there are not "this is an emergency, please land yourself" buttons for smaller aircraft:
* https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-...
They do if they're crossing railroad tracks.
That’s why I said 30 seconds / 30 minutes not 3 seconds. The idea is to go to somewhere safe and pull off the road not just slam on the breaks and hope your not on a curve.
* train tracks as mentioned by another comment
* getting out of the way of emergency vehicles or following instructions from emergency workers
I don’t think this detracts from the overall point—if the uptime is very good then the times when LiDAR doesn’t work and the car needs to move might practically never happen. Or the car might be able to move forward to a safe position in some limp-along mode.
WRT “this is an emergency drive to a hospital without LiDAR,” I think that would be pretty bad to include. What exactly qualifies as an emergency (this will be abused by some users)? And anyway, in most cases it is better to have an ambulance for an emergency. Finally, an emergency on my part doesn’t entitle me to endanger society. Rather, the types of behavior that the car can perform without all the sensors should be kept track of. If the car can limp along without LiDAR, then that’s something it can do (with the caveat that some roads are not safe to drive far under the speed limit on).
At minimum cars need to handle being on a curve when LiDAR or vision etc cuts off. Stopping on a freeway or railroad is a real risk even if people can get out, but now you’re doing risk mitigation calculations not just trying to handle normal driving. “What if someone is asleep in a level 5 car?” is a question worth considering, but we’re a long way self driving without any from of human backup.
The hospital thing is for the very real possibility of being outside of cellphone range, and represents the apex of gracefully shutting down the ride vs refusing to travel while in a less safe condition.
A self driving Taxi with a tire pressure sensor error vs total break failure are simply wildly different situations and each should get different responses. Further, designing for a possibility sleeping but valid human driver is very different than designing for a likely empty vehicle.
If you aren’t mitigating it with the appropriate controls, you aren’t managing risk. My point is just passing the buck to the human is not an appropriate control in many critical scenarios.
No. A break failure doesn’t guarantee a specific negative outcome, it dramatically raises the probability of various negative consequences.
My point is risk mitigation is about lowering risk but there may be no reasonably safe options. A car stopped on a freeway is still a high risk situation, but it beats traveling at 70 MPH without working cameras.
I agree there may be cases where there are no reasonably safe options. That means your engineered system (especially in a public-facing product) is not ready for production because you haven't met a reasonable risk threshold.
> I agree there may be cases where there are no reasonably safe options. That means your engineered system (especially in a public-facing product) is not ready for production because you haven't met a reasonable risk threshold.
Individual failures should never result in such scenarios, but ditching in the ocean may be the best option after a major fuel leak, loss of all engine power, etc.
I don’t think any FMEA is going to list “ditch into the ocean” as an acceptable mitigation. Ie it will never be a way to buy risk down to an acceptable level.
>it isn’t an analysis of mitigation strategies.
Take a look at NASAs FMEA guidebook. It clearly lists identifying mitigations as part of the FMEA process. You’ll see similar in other organizations but possibly with different names (“control” instead of “mitigation”)
https://standards.nasa.gov/sites/default/files/standards/GSF...
100% AI systems can be safer than human in the loop systems avoiding suicide by pilot etc, but conversely that means the AI must also deal with extreme edge cases. It’s a subtle but critical nuance.
An FMEA (and other safety-critical/quality products) go through an approval process. So if "ditch into the ocean" is not on the FMEA, it means they should have other mitigations/controls that bought the risk down to an acceptable level. They can't/shouldn't just push forward with a risk that exceeds their acceptable tolerance. If implemented correctly, the FMEA is complete insomuch as it ensured each hazard was brought to an acceptable risk level. And certainly, a safety officer isn't going to say the system doesn't need further controls because they put "ditch into the ocean" in the manuals. If that's the rationale, it begs the question "Why wasn't the risk mitigated in the FMEA and hazard analysis?" Usually it's because they're trying to move fast due to cost/schedule pressure, not because they managed the risk. There are edge cases, but even something like a double bird strike can be considered an acceptable risk because the probability is so low. Not impossible, but low enough. That’s what “ditch in the ocean” operations are for.
I agree that software system can improve safety but we shouldn't assume so without the relevant rigor that includes formal risk mitigation. Software tends to elicit interfacing faults; the implication being as the number of interfaces increases the potential number of fault modes can increase geometrically. This means it is much harder to test/mitigate software faults, especially when implementing a black-box AI model. My hunch is that many of those trying to implement AI in safety-critical applications are not rigorously mitigating risk like more mature domains. Because, you know, move fast and break things.
This is not how flying works in a multi-crew environment. It’s a common misconception about the dynamic.
Both pilots have active roles. Pilots also generally take turns who is manipulating the flight controls (“flying the airplane”) each flight.
Don't think that's the right analogy. Realistically you'd aim to combine them meaningfully. A bit like two eyes gives you depth perception.
You assume 1+1 is less than two, when really you'd aim for >2
If you have two sensors, one says everything's fine but the other says you're about to crash, which one do you trust? What if the one that says you're about to crash is feeding you bad data? And what if the resulting course correction leads to a different failure?
I'd hope that we've learned these lessons from the 737 Max crashes. In both cases, one sensor thought that the plane was at imminent risk of stalling, and so it forced the nose of the plane down, thereby leading to an entirely different failure mode.
Now, of course, having two sensors is better than just having the one faulty sensor. But it's worth emphasizing that not all sensor failures are created equal. And of course, it's important to monitor your monitoring.
Neither. You stop the car and tow it to the nearest repair facility. (Or have a human driver take over until the fault is repaired.)
You don’t gouge out your ears because you hear something walking around at night that doesn’t appear to be accurate. As a human, your executive function makes judgements based on context and what you know. Your eyesight is degraded in the dark, so your brain pays more attention to unexpected sound.
The argument for lidar, sonar or radar isn’t that cameras are “bad”, it’s that they perform very well in circumstances where visual input may not. As an engineer, you have an ethical obligation to consider the use case of the product.
It’s not at all like the 737max issue - many companies have been able to successfully implement these features I have a almost decade old Honda that uses camera and radar sensors to manage adaptive cruise and lane keeping features flawlessly.
In the case of Tesla and their dear leader, they tend to make dogmatic engineering decisions based on personal priorities. They then spackle in legal and astroturf marketing bullshit to dodge accountability. The folly of relying on cameras or putting your headlights inside of a narrow cavity in the car body (cybertruck) is pretty obvious if you live in a place that has winter weather and road salt.
I agree that optimizing for vehicle cost and shipping shoddy software at the expense of human lives is the wrong tradeoff, and having different sensors that provide more coverage of situations is generally preferable if you can afford it.
But suppose you have a radar sensor that's faulty, and so it periodically thinks that there's something right in front of it, and so it causes the car to slam on its brakes. That's likely going to cause an accident if you're traveling at highway speeds. Does that mean that we shouldn't use radar in any circumstances? Of course not. But sensors are not infallible, and we need to keep that in mind when designing these systems.
If you’re Waymo, you’ve built in a higher standard of sensor and diagnostics, at the expense of the aesthetic.
That’s always been the issue with Tesla… they push 99% solutions for 99.99% problems.
That's fair. I was likely remembering that there were two sensors, since there's one for the pilot and one for the copilot, but not remembering that only one of them was in use.
The other lessons worth noting are that 1) you need to give the human some indicator that something's wrong, and 2) you need to give the human some way of overriding the system.
> And unlike planes, you can safely stop a car in almost all scenarios.
Agreed, mostly. We do have to be careful about how the vehicle disengages and comes to a safe stop, or else we risk repeating the incident where Cruise hit a pedestrian, then pulled to the side of the road while dragging the pedestrian under the vehicle.
Maybe I'm just old, but development roadmaps are always vapor, until they aren't, which happens sometimes. Always been that way.
To be clear, I'd much rather my vehicle have superhuman multisensory awareness than only superhuman awareness. And I think it's fair for regulators to involve themselves with vehicle engineering, as all our safety depends on it. I've also watched the AI day presentations about their vehicle training system, and read their disclaimer text for enabling FSD, and it seems like they're doing a lot to advance the state of the art.
And not just a little bit - it’s way overpriced.
There are a bunch of possible explanations for this. One is that investors believe full self driving will come out really soon and work really well.
I own a 1978 Suzuki Carry and a Miles Electric ZX40ST.
I have watched a fair amount of https://www.youtube.com/@MunroLive with interest about the implementations from all manufacturers. Sandy is in my home state of Michigan, birthplace of the auto industry, in which I've been multi-generationally involved, and he knows his stuff. He has criticisms for all manufacturers, but over the years Tesla seem to have listened more than most, to the point of Elon speaking for hours with Sandy on podcasts about technical aspects of the vehicles and production. I also appreciate that Tesla seem to make more of their cars in the US than any other manufacturer. Honda seems to be the only one comparable. I think the future's electric. I don't really care who makes it, but I'd like it to be well engineered, and made locally.
I'd personally probably think traditional ICE manufacturers and oil industries are overvalued, but I'm probably wrong as there's clearly lots of business for those industries which doesn't seem to be going anywhere.
In medicine, the ethics commitee would shot the project early.
Edit: Comparing "ev vehicle" with "autonomous vehicle"
It seems like Tesla's been pretty clear with drivers who've opted in to FSD that they're helping to test and develop a system which isn't perfect.
Just off the top of the head, look up Henrietta Lacks for a notable example of how medicine has handled informed consent.
Strawman No matter how many contracts and disclaimers they get in their favor. The "Full self driving" system is causing accidents (and deaths)
Turn lights, rearview cameras, safety belts, all exist and are regulated to prevent the manufacturer dumping responsibility on the drivers.
I emphasize "Full self driving" Tesla brand because the name was declared unlawful in California. https://www.govtech.com/policy/new-california-law-bans-tesla...
Well yeah, and so is every other driver on the road. That is not the relevant metric. The real question is whether or not it is safer than the average driver. Or safer than great aunt Marge who doesn't have the best eyesight or hearing, but somehow still has a drivers license.
https://www.wired.com/story/lidar-cheap-make-self-driving-re...
It's more like, a pilot has access to multiple sensors. Which they do.
more likely
he is a ruthless ** who doesn't care about people dying and slightly increasing the profit margin is for him worth more then a some people dying
regulators/law allowing self driving companies to wriggle out of responsibility didn't help either
lidar was interesting for him as long as it seemed light Tesla could maybe dominate the self driving marked through technological excellency the moment it was clear it won't work he abandoned technological excellence in favor of micro optimizing profit at the cost of read safety
which shouldn't be surprising for anyone I mean he also micro optimized workplace safety in SpaceX away not only until it killed someone but even after it did (stuff like this is why there where multiple investigations against his companies until Trump magiced them away)
the thing is he has intelligent people informing him about stuff, including how removing lidar will statistically seen kill people, so it's not that he doesn't know, it's that he doesn't care
Someone said in an interview “Elon desperately wants the world saved. But only if by him.”
I believe that's also Lex Luthor's motivation in All Star (?) Superman.
There is no planet B.
The same argument likely applies to you (assuming you are in a wealthy nation) so you are likely just as ruthless::
We could all slightly decrease our disposable incomes (spent on shit we don't need) and increase the life expectancy or QoL for someone in a poor country.
Me too: I spent thousands on a holiday (profit for my soul) and I didn't give the money to a worthy charity. I'm no fan of Musk, but I think there's better arguments for dumping on him if you really need to do that.
I also never said there aren't many other things bad about Musk.
And comparing doing decisions out of greed which will straight forward risk the live of many and you aren't even gaining that much from it (compared with what you already do gain from the same source) with "we as a society could if we where a hive mind life more frugal and help another society else where" is just a very pointless thing.
But even if they where the same, what does that change, just because you also do something bad doesn't mean it's less bad or more tolerable or should be tolerated.
> We could all slightly decrease our disposable incomes (spent on shit we don't need) and increase the life expectancy or QoL for someone in a poor country.
But we can't, or more specifically any specific individual can't, they can at best try to influence things by voting with their money and votes. But that is a completely different context.
And that doesn't mean you shouldn't spend you money with care and donate money if you can afford it.
Just see him talking about things at NeuraLink. Musk wouldn't exist if it weren't for the people working for him. It's a clown that made it to the top in a very dubious way.
In reality getting rich has more to do with opportunity, connections, and luck - but accepting that means you've got to accept that the American Dream has always been a lie. It's much easier to convince yourself that people like Elon are geniuses.
The others don't claim the extremes of power and genius, based to a large extent on what their teams do. They also build good teams - look at DOGE, for example.
I've decided Musk core talent is creating and running an engineering team. He's done it many times now - Tesla, SpaceX, Paypal, even twitter.
It's interesting because I suspect he isn't a particular good engineer himself, although the only evidence I have for that is tried to convert Paypal from Linux to Windows. His addiction to AI's getting results quickly isn't a good look either. To make the product work in the long term the technique has to get you 100% of the way there, not the 70% we see in Tesla and of now DOGE. He isn't particularly good at running businesses either, as both twitter shows and his solar roof's show.
But that doesn't matter. He's assembled lots of engineering teams now, and he just needs a few of them to work to make him rich. Long ago it was people who could build train lines faster and cheaper than anyone else that drove the economy, then it was oil fields, then I dunno - maybe assembly lines powered by humans. But now wealth creation is driven by teams of very high level engineers duking it out, whether they be developing 5G, car assembly lines or rockets. Build the best team and you win. Musk has won several times now, in very different fields.
So if your goal is to pump out $20k self driving cars, then you need cameras to be good enough. So the logic becomes "If humans can do it, so can cameras, otherwise we have no product, no promise."
More importantly, expectations are higher when an automated system is driving the car. It is not sufficient if, in aggregate, self-driving cars have fewer accidents. If you lose a loved one in an accident where the accident could have been easily avoided if a human was driving, then you're not going to be mollified to hear that in aggregate, fewer people are being killed by self-driving cars! You'd be outraged to hear such a justification! The expectation therefore is that in each individual injury accident a human clearly could not have handled the situation any better. Self-driving cars have to be significantly better than humans to be accepted by society, and that means it has to have better-than-human levels of vision (which lidars provide).
Put another way - would giving humans superhuman vision significantly reduce the accident rate?
The issue here is that the vision based system failed to even match human capabilities, which is a different issue from whether it can be better than humans by using some different vision tech.
Yes? Incredibly?
If people had 360° instant 3D awareness of all objects, that would avoid so many accidents. No more blind spots, no more missing objects because you were looking in one spot instead of another. No more missing dark objects at night.
It would be a gigantic improvement in the rate of accidents.
They don’t leave enough space/time to react even if they did have enough awareness
If you assume a constant rate of attention, but then massively increase what people are aware of during that attention, that's a massive increase in safety.
Safety is affected by lots of factors. Increasing any of them increases safety -- you don't need to increase all of them to get improvements.
Many people are in accidents that are entirely not their fault, e.g. being rear-ended. You can't do anything about those who drive unsafely, distracted, under the influence, etc.
Assuming you have a "good" driver, there are still plenty of times they might be into accidents for reasons that aren't because of awareness. For example, road conditions, behavior of other drivers, and just normal, honest mistakes.
At least for me, my own sense aren't really the limiting factor in my safety. My eyes are good enough. Modern cars have the equivalent of bowling lane bumpers with auto-centering, pre-collision warning, and blind spot warning/indicators.
Your eyes aren't good enough, compared to if you had 360° LIDAR awareness, which is the comparison here.
You're listing all these other causes of accidents, which no one disputes. There's still a whole range of accidents caused by limitations in our spatial awareness because we can only ever be looking in one direction at a time.
You're talking about "normal, honest mistakes". That's what I'm talking about. There would be less of those if we had magic 360° LIDAR in our bodies. A lot less. Especially at night, but during the day too.
This isn't about blaming anybody. It's just the simple scientific fact that human senses are limited, and self-driving cars shouldn't limit themselves to human senses. Human senses aren't good enough for preventing all preventable accidents, no matter how "honest" drivers are.
> Put another way - would giving humans superhuman vision significantly reduce the accident rate?
I'm saying that human vision is not the cause of most accidents. Most accidents are caused by distraction, incorrect decisions/errors of judgement, road conditions, etc.
You're using existing human vision as your baseline, which is not the right comparison here. The question is a baseline of superhuman abilities. So there's a ton of accidents we wouldn't get into if we had those superhuman abilities.
Basically everything involving a collision with an object we weren't aware of in time because it wasn't in our limited field of view and wasn't well-lit, that could have been avoided if we had been. Which is a decent proportion of accidents.
But the bigger, original point is that LIDAR is better than cameras, and better for avoiding accidents.
Of these, only dark objects at night is related to lidar vs vision. What percent of accidents is attributable to not seeing dark objects at night?
Limited field of view is solved by having a bunch of cameras, that is what is orthogonal to LiDAR vs vision.
I’m open to an argument here, but you have not provided any compelling rebuttal. You apparently feel like you have, though.
You use some made-up idea like "superhuman vision" and then you're asserting that I haven't provided a "compelling rebuttal"?
You're not engaging in good-faith conversation here. If you want to understand the clear, obvious benefits of LIDAR over cameras, it's a Google search away. And it's not just about at night -- it's about weather, it's about greater accuracy, especially at greater distances, it's about more accurate velocity, and so forth. It's not rocket science to understand that earlier, more confident detection of a car or deer or child suddenly moving into the street is going to reduce collisions.
Obviously LiDAR improves the maximum achievable capabilities of a self driving system. I am not disputing that. I don’t think anyone would or could dispute that, it’s just trivially true.
The point I am trying to make is more nuanced than that.
Basically I’m thinking about whether this decision to ditch lidar is actually stupid, or if there could be a sensible explanation that is plausibly true. I am proposing what I believe is a plausibly true sensible explanation for the decision.
If you think about whether accidents would be reduced by having lidar, the answer is again obviously yes. But if you think about this as a business decision, it is not so simple. The question is not whether accidents would be reduced by having lidar. The question is whether accidents would be reduced enough to justify the added cost and complexity to the system.
So then, how do we figure that out? Well, we think about the causes of human car accidents ranked by percent of car accidents caused. What would be the items in the top 95% let’s say? Then, which of those items can be addressed exclusively by a lidar enabled system.
To isolate can-only-be-fixed-by-lidar problems, I imagine a thought experiment where we have a human with lidar for eyeballs. Which of the top k causes of accidents would be fixed by giving humans lidar eyeballs. Well, maybe not that many, actually. That is my point.
This could be argued to be immoral due to I guess making decisions about death based on business considerations. But it’s probably fairly easy to convince oneself as a Tesla executive that the car (M3 anyway) being affordable is critical to getting the safer-than-human driving tech adopted, so making the car more expensive actually fails to save more people than the lidar would save, etc.
Looking in one spot instead of another is included in what I’m calling “attention”. Of course paying attention to everything all the time would be and is a huge improvement. That is orthogonal to the type of vision tech being used. All approaches used in self driving systems today look everywhere all the time.
So I'm assuming that the "superhuman vision" you described meant specifically if people had LIDAR, since that was the subject at hand.
LIDAR is superior to cameras because it much more accurately detects objects. It's an advantage all the time, and an especially huge advantage at night.
So LIDAR isn't orthogonal to anything. It's the entire point. If people had LIDAR, the accident rate would be significantly reduced.
You're arguing that LIDAR won't help cars, because by analogy it wouldn't help people if it was a native biological sense. But that's wrong.
Never driven before?
Maybe if your standard is "scheduled commercial passenger flights" level of safety.
>Otherwise why not just drive the car yourself?
There's plenty of reasons why humans can be more dangerous outside of vision quality. For instance, being distracted, poor reaction times, or not being able to monitor all angles simultaneously.
As a pitch for self driving, it's going to be a long time before I trust a computer to do the above better than I do. At the very least adding sensors I don't have access to will give me assurance the car won't drive into a wall with a road painted on it. I don't know how on earth you'd market self-driving as competent without being absurdly conservative about what functionality you claim to be able to deliver. Aggregate statistics about safety aren't going to make me feel emotionally stable when I am familiar with how jerky and skittish the driving is under visually confusing driving conditions.
Perhaps vision is sufficient, but it seems hopelessly optimistic to expect to be able to pitch it without some core improvement over human driving (aside from my ability to take the hand off the wheel while driving).
Edit: hilariously, there's already a video demonstrating this exact scenario: https://youtu.be/IQJL3htsDyQ
This is as relevant as self driving cars not being able to detect anti-tank mines. If you want to intentionally cause harm, there are far easier ways than erecting a wall in the middle of a roadway and then painting a mural on it. If you're worried about it accidentally occurring, the fact that there's no incidents suggests it's at least unlikely enough to not worry about.
>Aggregate statistics about safety aren't going to make me feel emotionally stable when I am familiar with how jerky and skittish the driving is under visually confusing driving conditions.
Sounds like this is less about the tech used (ie. cameras vs lidar) and how "smooth" the car appears to behave.
Driving requirements in many states demand 20/40 vision in at least one eye [1]. 20/20 visual acuity is a arc-resolution of approximately 1 arc-minute [2] thus 20/40 vision is approximately a arc-resolution of 2 arc-minutes or 30 pixels per degree of field of view. Legally blind is usually cited as approximately 20/200 which is approximately 10 arc-minutes or 6 pixels per degree of field of view.
Tesla Vision HW3 contains 3 adjacent forward cameras at different focal lengths and Tesla Vision HW4 contains 2 adjacent forward cameras of different focal lengths and as such those cameras can not be used in conjunction to establish binocular vision [3]. As such, we should view each camera is a zero-redundancy single-sensor and is thus a "single-eye" case.
We observe that Tesla Vision HW3 has a 35 degree camera for 250m, 50 degree camera for 150m, and 120 degree camera for 60m [4]. The Tesla Vision HW4 has a 50 degree camera for 150m, and 120 degree camera for 60m [4]. A speed of 100 km/h corresponds to ~28 m/s as such the cameras trailing times of ~10s, ~6s, ~2s. Standard safe driving practices dictates a 2-3 second follow, so most maneuvers would be dictated by the 60m camera and predictive maneuvers would be dictated by the 150m camera.
We observe that the HW3 forward cameras have a horizontal resolution of 1280 pixels resulting in a arc-resolution of ~25.6 pixels per degree for the 150m camera and ~11 pixels per degree for the 60m camera, the camera used for the majority of actions. Both values are below minimum vision requirements for driving with most states with the wide angle view within a factor of two of being considered legally blind.
We observe that the HW4 forward cameras have a horizontal resolution of 2896 pixels resulting in a arc-resolution of ~58 pixels per degree for the 150m camera and ~24 pixels per degree for the 60m camera. The 60m camera, which should be the primary camera for most maneuvers, fails to meet minimum vision requirements in most states.
It is important to note that there are literally hundreds of thousands if not millions of HW3 vehicles on the road using sensors that fail to meet minimum vision requirements. Tesla determined that a product that fails to meet minimum vision requirements is fit for use and sold it for their own enrichment. The same company that convinced customers to purchase systems when they promised: "We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."[5] in 2016 when they were delivering HW2. Despite the systems being delivered clearly not reaching even minimum vision requirements and, in fact, being nearly legally blind.
[1] https://eyewiki.org/Driving_Restrictions_per_State
[2] https://en.wikipedia.org/wiki/Visual_acuity
[3] https://en.wikipedia.org/wiki/Tesla_Autopilot_hardware
[4] https://www.blogordie.com/2023/09/hw4-tesla-new-self-driving...
[5] https://web.archive.org/web/20240730071548/https://tesla.com...
This isn't as much as a slam dunk as you think it is. The fallacy is that assuming visual acuity requirements are chosen because they're required for safe maneuvering, when in reality they're likely chosen for other tasks, like reading signs. A Tesla doesn't have to do those things, so it can potentially get away with lower visual acuity. Moreover if you look at camera feeds from HW3/HW4 hardware, you'll see they're totally serviceable for discerning cars. It definitely doesn't feel like I'm driving "legally blind" or whatever.
There is no affordable vision system that's as good as human vision in key situations. LiDAR+vision is the only way to actually get superhuman vision. The issue isn't the choice of vision system, it's to choose vision itself, and besides the lesson from the human sensory system is to have sensors that go well with your processing system, which again would mean LiDAR.
If humans could integrate a LiDAR-like system where we could be warned of approaching objects from any angle and accurately gauge the speed and distance of multiple objects simultaneously, we would surely be better drivers.
Yeah, he was arguably wrong about one thing so his building both the world's leading EV company and the world's leading private rocket company was fake.
As they say, the proof of the pudding is in the eating. Between Tesla, SpaceX, and arguably now xAI, the probability of Musk's genius being a fluke or fraud is close to zero.
But, he is frequently wrong, it just does not matter. He was occasionally right, like with Tesla back then.
We already know he's an objective fraud because he literally cheats at video games and was caught cheating. As in, he hired people to play for him and then pretended the accomplishments were his own. Which maps very well to literally everything he's done.
To be fair, the ghost brakes on TACC have reduced. But I tend to control my wipers with voice.
But why make the jump to "fraud/charlatan"? Every system needs to be finite. We can invest in every bell and whistle. Furthermore, he's upfront about the decision. Fraud requires deception.
So, I was deceived. I didn't buy the car because of the deception, but I did buy FSD because of it.
Also, FSD disengaging when it gets sensor confusion should be considered criminal fraud. FSD should never disengage without a driver action.
Instead of critiquing the article for liberal use of words like "overwhelmingly", "unique", and " 100% of Teslas" on an n=5 cars, with limited data and a very questionable analysis of the Snohomish accident, we discuss how Musk is a fraud.
That was Karpathy's decision [1] and, yes, I also have that perception of him.
I know this is not going to be well received because he's one of HN's pet prodigies but, objectively, it was him.
1: https://www.forbes.com/sites/bradtempleton/2022/10/31/former...
(one of many)
I read that as Musk wanted this done and asked Karpathy to find a way.
Seems to me that the guy is pretty convinced, but sure your mind belongs to you and you can make it whatever you want, maybe Musk was behind with a gun, who knows.
Why? Because it provided information that people had to infer, and that you couldn't easily get from camera. So absent the human inference engine that allowed human drivers to work, we would have to rely on highly-precise measurement instruments like LiDAR instead.
Musk's huge error was in thinking "Well humans have eyes and those are kind of like cameras, therefore all you need are cameras to drive"
But no! Eyes are not cameras, they are extensions of our brains. And we use more than our eyes to navigate roads, in fact there's a huge social aspect to driving. It's not just an engineering challenge but a social one. So from the get-go he's solving the wrong problem.
I knew this guy was full of it when he started talking about driverless cars being 5 years out in 2015. Just utter nonsense to anyone who was actually in that field, especially if he thought he could do it without LiDAR. He called his system "autopilot" which was deceptive, but I was completely off him when he released "full self driving - beta" onto public streets. Reckless insanity. What made me believe he is criminally insane is this particular timeline (these are headlines, you can search them if you want to read the stories):
2016 - Self-Driving Tesla Was Involved in Fatal Crash, U.S. Says
2016 - Tesla working on Autopilot radar changes after crash
2017 - NTSB Issues Final Report and Comments on Fatal Tesla Autopilot Crash
2019 - Tesla didn’t fix an Autopilot problem for three years, and now another person is dead
2021 - Inside Tesla as Elon Musk Pushed an Unflinching Vision for Self-Driving Cars Tesla announces transition to ‘Tesla Vision’ without radar, warns of limitations at first
2022 - Former Head Of Tesla AI Explains Why They’ve Removed Sensors; Others Differ
2022 - Tesla Dropping Radar Was a Mistake, Here is Why
2023 - Tesla reportedly saw an uptick in crashes and mistakes after Elon Musk removed radar from its cars
2023 - Elon Musk Overruled Tesla Engineers Who Said Removing Radar Would Be Problematic: Report
2023 - How Elon Musk knocked Tesla’s ‘Full Self-Driving’ off course
2023 - The final 11 seconds of a fatal Tesla Autopilot crash
The man and his cars are a menace to society. Tesla would be so much further along on driverless cars without Musk.
Besides, rangefinding optics weren't exactly amazing in the 1940s either. It's why the introduction of radar made such a massive difference in naval warfare.
I'll reiterate:
2019 - Tesla didn’t fix an Autopilot problem for three years, and now another person is dead
> And if it was truly unsafe or compromised we'd have seen a good deal of evidence
We see that evidence all the time. Teslas veering into oncoming traffic, hitting parked vehicles, driving through loony tunes walls where other cars stop, being fooled by smokescreens where other cars are not, and decapitating multiple people in a similar way that would have been mitigated by LiDAR. And now apparently real-ending motorcyclists.
This whole self driving scam is an exercise in the 80/20 rule. They spent 20% of the time to get 80% of the results, and that's why you claim "I see teslas self driving just fine"
But Tesla has been promising the other 20% will be here in 5 years for 10 years. That last 20% is the difference between the system being "Full" self driving and a fraud. Right now what we see is a vaporware fraud, and they're not going to be able to deliver.
You are the one who raised the 1940s optical rangefinder question, and now talking about autofocus, which is a totally different problem than dense ranging from optical stereo disparity.
If you don't want to argue hypotheticals, don't do it.
If you choose to argue technical details in the absence of expertise, expect pushback from people with domain knowledge.
I spent 5 years working on long-distance optical stereo as part of a DARPA program, with a team that was very committed to stereo vision. Our limited success left me with an appreciation of the challenges versus lidar.
Second, what poisons the idea among engineers is not that Musk came up with it, but that he can't justify it using the language we speak, either by theory or by practice. Instead what we get from him is marketing, stunts, keynotes, sleights of hand, bravado, exaggerations, unfulfilled promises, premature rollouts of untested tech... the man is an anti-engineer, it's hard to trust him.
What made cars successful at the DUC was omnidirectional 3D distances to everything around you provided by the Velodyne LiDAR. So if we have a way to get that kind of data without a LiDAR, that would be fine.
Moreover, what a LIDAR gives you is an honest-to-god measurement. This whole idea of using deep learning to get range data from camera data is not measuring anything, it's making an inference. Which is why it's fooled by a looney tunes wall.
And like I said, we lack any inference engine that is better than the human brain. So betting your entire strategy on an inference engine that doesn't exist, bucking industry practice and the consensus of the engineering community, gets the following results:
- The NHTSA’s self-driving crash data reveals that Tesla’s self-driving technology is, by far, the most dangerous for motorcyclists, with five fatal crashes that we know of.
- This issue is unique to Tesla. Other self-driving manufacturers have logged zero motorcycle fatalities in the same time frame.
- The crashes are overwhelmingly Teslas rear-ending motorcyclists.The error on range estimates from stereo “disparity” goes up as range squared for fundamental-physics reasons. Accurate calculation of disparity relies on well calibrated optics (clean, physically rigid) and it’s easy to disrupt that.
Stereo is also target-sensitive. Stereo ranging is enhanced by certain target features, like large, smooth surfaces with texture (say, stucco walls, or car grilles). It is made more difficult by smaller target surfaces with weird curvature.
I’m sure that stereo that did well on auto-sized surfaces would do much worse for pedestrian or motorcycle (size and shape) surfaces.
https://www.bizjournals.com/sanjose/news/2022/11/09/heres-wh...
He began selling shares because he didn't like the direction of the company and was unhappy about being forced out of it, he told the Business Journal. From almost the moment he was pushed out until this spring, he fought a war with Velodyne in the press, at shareholder meetings and in the courts. Although the lawsuit he filed against the company is ongoing, he eventually decided the company was a "dump" and couldn't be salvaged.
[0] https://x.com/SethAbramson/status/1892710698683142638Computer Vision has turned out to be a very tough nut to crack and that should have been visible from anyone doing serious work in the field since at least 15 years ago.
In any case, any safety critical system should be build with redundancy in mind, with several sub systems working independently.
Using more and better sensors is only a problem when building a cost sensitive system, not a safety critical one, and very often those sensors are expensive because they are niche, that can be mediated with mass scale.