The fact that they profit handsomely off this structurally dangerous wrongdoing is just the cherry on top.
And, without robustly maintaining a systems safety lifecycle (which, by necessity, must incorporate a Safety Management System)... no technical progress is quantifiable by anyone, including Tesla.
Tesla effectively throws a system over-the-wall and throws it all on the human driver and on the public.
You do understand that people are concerned about setting these cars free on public roads, right, where they can kill unwilling participants? I don't think the concern here is about your freedom to choke on billionaire boots.
Elon is a bit of a monster (personal opinion), but regulators have the final say. When they force FSD to be pulled, then there is weight behind the argument, but this hasn’t happened and that sends signal.
You already share the road with inattentive drivers and drunks, so the risk acceptance/appetite benchmark has been set. FSD is arguably better than both cohorts, considering number of deaths caused.
As always, nuance. More people will die in the next ~10 minutes from traffic deaths than have ever been attributed to Tesla’s autopilot or FSD (33 total, as of this comment).
(Again, not a billionaire simp, just a rationalist; booo on Elon, but props to Tesla engineers in the aggregate; personally, I hope he gets blown out the door and JB Straubel takes over as CEO)
This is true.
> When they force FSD to be pulled, then there is weight behind the argument.
Well, I suppose that it is pretty hard to dispute this, but it should be recognized that the NHTSA (the theoretical regulator of vehicle and highway safety in the US) is extremely weak and virtually non-existent.
The NHTSA lacks anything close to the skill sets necessary to independently, proactively and robustly scrutinize even rudimentary mechanical issues (which has been confirmed by several USDOT OIG reports over the years).
With opaque, complex automated systems and software... the NHTSA stands no chance.
The NHTSA lacks the internal skill sets to understand any of the comments that I have made elsewhere on this post.
Again, you are not wrong per se, but again, it should be recognized that the NHTSA is concerned primarily with establishing plausible deniability to protect the agency and with headlines rather than protecting the public with solid regulatory processes and oversight.
(Coincidentally enough, yet another USDOT OIG report was buried in a Friday afternoon release: https://www.autoblog.com/2023/06/02/nhtsa-fails-to-meet-inte.... I kid you not, every four years or so the USDOT OIG releases another critical report on the NHTSA that focused on issues not rectified in the previous report. It is like Groundhog Day.)
> You already share the road with inattentive drivers and drunks, so the risk acceptance benchmark has been set.
This is true.
Because the US public does not demand change and because roadway deaths are high, but distributed across time and space... the NHTSA remains weak and overall US transportation policy remains dreadfully poor.
> FSD is arguably better than both cohorts, considering number of deaths caused.
Unquantifiable.
There is no way to accurately and independently quantify the downstream safety impact of FSD Beta.
Sure, perhaps the NHTSA believes that (because they must given their structural issues), but we should recognize why such assumptions are flawed.
> More people will die in the next few minutes from traffic deaths than have ever been attributed to Tesla’s autopilot or FSD (33 total, as of this comment).
There is the possibility for "indirect" incidents caused by FSD Beta where the FSD Beta-active vehicle is never physically impacted.
We cannot assume that those do not exist.
And we also cannot assume that the media is able to pick up on every Tesla vehicle-related incident - even as well-followed as Tesla, the company, is.
In fact, other than the automaker's word, in many cases, safety investigators like those from the NTSB cannot independently and forensically establish specific root causes.
The second is a crime and I believe the first is a misdemeanor. Getting caught in either scenario repeatedly will cause you to lose your license.
So we may share the road with dangerous drivers, but we don't accept it. So it isn't grounds to accept more danger.
Really this line of argument is always wrong. The presence of danger should make you less comfortable accepting additional danger, not more. It's not like they cancel out, they sum. (One might say that mature and accessible self driving technology would take these drivers off the road, but the situation today is immature technology and high end vehicles.)
As a self described rationalist, I think you should take another look at that - to me, it reads like you're saying that because it doesn't feel like we're taking on additional marginal risk in comparison to the risks we've already taken on, we don't need to worry about how we're actually doing so, so I was caught a bit off guard when you said you were a rationalist.
I'm skeptical of any digital technology use case where the analogies/comparisons tend to be:
a) to non-digital things, when there are perfectly decent digital comparisons to be made (e.g., autopilot in the airline industry)
b) to conspicuous non-digital things, here drunk or inattentive humans
c) more or less a $small-human-scale-X improvement on the performance/efficiency/safety of the non-digital thing, often implying a future $unspecified-X improvement that, say, Moore's law would suggest (not saying you're doing the latter here, btw)
Those last two especially. I'm just imagining someone hawking a newfangled realtime audio system, claiming that it outperforms hand-punching a player-piano score. It's a silly example for sure; but on a regular basis on HN I read how Bitcoin is no worse than fiat in terms of global energy use, how crypto scam rates are roughly equivalent to wildcat banks in the old west (and hey, we eventually improved on those, so...), and how many more humans cause car deaths than these intractable systems which are "arguably" better than drunks.
[1] https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
And yet, the average driver kills 0 people. It’s in the far margins where deaths occur - a fraction of a fraction of a percent.
This is in comparison to the arguably singular entity “Tesla Autopilot” which has already killed several.
And we hold them responsible for the humans in their care.
The fact that Tesla does not put their money where their mouth is says all you need to know.
Wake me up when someone is willing to take on liability. Until then automated driving does not exist.
FSD with a safety driver does not pose a threat to public safety. That’s not coming from me, it’s coming from the NHTSA.
I would have thought that something called Full Self Driving would be capable of fully driving by itself.
Sarcasm aside, that’s how most people use it.
The NHTSA hasn’t regulated it, but that doesn’t mean it’s an endorsement. It just means that they haven’t found anything within their legislative mandate that they can regulate here. US agencies are largely reactive, and I’m not sure I trust them with my life to get this right.
No one who has used the software for more than a minute has any misconceptions about FSD.
But people who have heard the name might. Honestly, it's a ridiculous name. Just "Self-driving" might be arguable. What are they going to call it when it's really fully self driving? "Literally Fully Self Driving Totally Serious"
Although, I guess my bigger issue is actually with the government that takes no action about false advertising.
My experience with FSD is it’s a terrible autonomous system and anyone who uses it as such is a fool, and a fool with a car is dangerous no matter what. However the joint probability of my driving awareness and skill and the cars combined is greater than mine alone. I’ve had it suddenly brake when a car I didn’t notice was drifting into my lane and had it not I would have been in an accident. Likewise it made mistakes and I took control.
I personally don’t care if it ever is able to take me from point A to point B without my attention or assistance. I value its ability to navigate with my assistance especially on long trips, reducing my overall fatigue and taking me through confusing sections of urban interstate without errors - when I always make a wrong turn. The fact it’s 360 aware and I’m not and it’s indefatigable and I’m not is valuable.
In the last year it’s become remarkably more capable. I don’t know if they can continue this rate of improvement but if they can it’s about as good as I would expect from todays technology on a consumer car. That’s a decent bar for me. I think it’s also something valuable on the roads - as a driver assistance tool. The folks who turn it on and get in the backseat would do something just as bone headed without FSD. Rather I notice enormous number of Tesla cars on the road not being drive by total idiots, and presumably quite a few using FSD without issue. And, as I assert above, I believe the joint probability of the aware driver with FSD having an accident is lower than either alone.
I don’t care what hyperbole a bipolar nut job spouts, but I do appreciate him setting an unreasonable goal and failing halfway there while the rest of the world seems content with stagnating. Tesla created the EV movement in the mainstream, SpaceX created the space revival we are experiencing.
Fwiw, I think the choking on billionaire boots comment is not a particularly high value contribution.
I do appreciate him setting an unreasonable goal and failing halfway there while the rest of the world seems content with stagnating.
I feel the same way about teenagers, grandmas, and grandpas, and yet here we are.
FSD does not have rights.
> first?
Yes.
According to your own citation, there are currently zero deaths where anyone has alleged any involvement of FSD Beta.
But even without that, there are no allegations of such FSD beta related fatal or incapacitating incidents, who would say it even if the beta put the human in harm’s way 10, 15 seconds earlier. The black box in the car includes both the dash cam camera and whether auto steer is activated, so any fatal crash would still leave such evidence of Beta being active.
Shouldn't the public be already outraging from the existing deaths? what's the difference... There will not be any change with an FSD death. As there wasn't with the previous deaths. Most people just don't care.
> So a Tesla driving off a cliff
This is a lie. There is no evidence that any Tesla vehicle has ever driven itself off a cliff.
Perhaps you are thinking about media reports from a few months ago where a human drove their vehicle off a cliff? If so, it is concerning that you misremembered and/or misrepresented this news event in this way. This news event was of a human driving a vehicle off a cliff. That vehicle happened to be a Tesla — noteworthy only because experts were saying that the chassis did an astoundingly good job of protecting its occupants.
No use of driving aids was ever alleged.
> Shouldn't the public be already outraging from the existing deaths?
Yes. A million people are killed by vehicles driven by humans every year. A million people dead. Every year. This is an outrage.
As for deaths where use of Tesla Autopilot is indicated or alleged, I find them no more outrageous than deaths involving any manufacturer's brand name for lane centering/adaptive cruise control. These features do not lower the driver's responsibility to keep the vehicle under control. And they do not diminish the accountability of the driver in the event of a collision.
I agree that where we are is balderdash, and dishonest, unclear about itself in extreme & lacking. But I detest this My Paradigm Is Required phrasing. Say what you think please! Browbeating the topic with particular/spefific engineering dogmatisms is unhelpful, unclear: it leans on authority, while also not having assertable claims anyone else could contest. This kind of hollow criticism degrades.
whether Tesla is level 2 or 4 autonomous. I'm very impressed with Tesla's FSD.
edit: Tesla FSD reach the destination first by taking the high way while Waymo go the local route and too far longer to reach the same destination.
https://www.youtube.com/watch?v=Hv9HtWUf27s
marques brownlee FSD demo: https://www.youtube.com/watch?v=9nF0K2nJ7N8
Google had a self driving car over 10 years ago, that is at the level of FSD, but their approach is to go straight to L5 for safety.
A Level 4-capable vehicle (a Waymo vehicle) is an incomparably different system than a Level 2-capable vehicle (a vehicle equipped with FSD Beta).
The Waymo vehicle has a design intent such that there is no human driver fallback requirement within their vehicle's Operational Design Domain (ODD).
The Tesla vehicle has a de facto design intent such that the human driver is the fallback at all times - which makes the control relationship between the human driver and the automated system exactly the same as if the Tesla vehicle was equipped with no automation at all.
The risk profiles and failure mode analyses are Night and Day different and, therefore, the validation traits between these two vehicle are Night and Day different.
But, more than that, there are no guarantees that:
- The human driver of the FSD Beta-active vehicle shown in that video did not manipulate any of the vehicle controls out-of-view that clandestinely assisted the vehicle without deactivating the automated system (possible and inherent Human Factors safety issues with that aside); and
- The creators of this comparison video did not select the most visually-performant run out of several attempts.
Naturally, since we are dealing with safety-critical systems here, assumptions of "positive safety" are not compatible with any internal or external analysis.
Lastly, I have yet to see a video involving FSD Beta where indirect and "unseen" systems safety issues were satisfied. Appearances can be deceiving and deadly with safety-critical systems.
Point is, you are assuming a certain legal outcome, and there is no reason to believe it won't go the opposite way
that's why i included Marques Brownlee's demo.
Videos and personal experiences can only reveal safety issues, never positive progress.
Marques (and every other FSD Beta user) is not read into a would-be systems safety lifecycle for this safety-critical systems that Tesla should be maintaining.
Marques (and every other FSD Beta user) is entirely blind to that.
It is a complete Black Box to them.
Therefore, the assessments made are always subjective and are almost entirely based on emotions and appearances (and other hand-wavy, ill-defined aspects such as "interventions" or "disengagements") rather than a complete accounting of all relevant systems safety components.
Systems safety is about exhaustively asking questions and then exhaustively seeking quantifiable answers to those questions against established failure modes and in the context of the system and every other system that interacts with it (including the human driver in the case of a FSD Beta-equipped vehicle as a Level 2-capable vehicle).
That is the whole point of a company maintaining a robust systems safety lifecycle - to convert subjective opinions of system characteristics into quantifiable understanding.
Tesla is not maintaining that.
Throughout the video, there are several places were Marques states "he thinks" or "he believes" or "that looks good" and such comments are also prevalent in the YouTube comments attached to the video.
These are safety-critical systems where an unhandled failure can readily result in an injury or death.
Responsible systems developers need something far more quantifiable than blind opinions of run-of-the-mill consumers.
That FSD Beta-active vehicles do not appear to "run into things as often" on the roadway is not a complete evaluation of the system.
There are also very real indirect and "unseen" safety components that are inherently part of the public roadways that must also be accounted for.
For what it is worth, I touched on some examples of this recently in a Mastodon thread: https://elk.zone/mastodon.social/@adamjcook/1101629508444173...
If you think it's a black box to the Tesla drivers, how is it not a black box to the Waymo customers in the back seat of these cars? Or how are you evaluating Tesla vs Waymo if not by how how humans subjectively feel each system is performing?
If you mean to the teams, you cannot assume that Waymo's systems are any less of a black box than Tesla's systems. And even then, they're not much of a black box at all, besides the actual object detection, as both Waymo and Tesla still have most decision-making in regular logic-based code, not machine learning algorithms; and when they do, such as with "do I need to get over now to make the next turn", it's still fed back into the "business logic" that decides what to do and thus logged and audited when it's sent back to HQ.
Also, I just realized that the systems safety engineer you responded to has also posted a reply, so you should look to their statement for a more in-depth analysis as they are a expert on the subject.
The general public (as a vehicle occupant) only interacts with a Waymo vehicle as a passenger with no vehicle control responsibilities.
That is in stark contrast from the integral human-machine relationship that exists in a Tesla vehicle.
> If you mean to the teams, you cannot assume that Waymo's systems are any less of a black box than Tesla's systems.
True.
Waymo's internal processes are a Black Box to me (and anyone external to Waymo) because we are not read into their systems safety lifecycle, whatever it may be.
Hopefully and presumably, Waymo is maintaining a Safety Management System (SMS) with their test operators and other internal teams, as they have claimed in the past.
Of course, since there is little-to-no regulatory oversight of this in the US (at the moment, perhaps)... Waymo's "word" is really the only thing the public has to go by.
That is not acceptable, in my view, in constructing a novel transportation system that ultimately relies on public trust to be economically viable... but that is the regulatory reality right now.
In the case of Tesla, it is definitive that they are not maintaining a SMS, in large part, because Tesla's (untrained) customers utilizing the system cannot be sufficiently read into a lifecycle. There is simply no way to do that without maintaining a highly-controlled, continuous relationship with the test operator.
For example, the "release notes" (sprinkled with some Tweets from Musk) that Tesla issues with some of the FSD Beta updates are simply too puny relative to the complexity of not only the vehicle system, but the larger complexity of the roadway.
> And even then, they're not much of a black box at all, besides the actual object detection, as both Waymo and Tesla still have most decision-making in regular logic-based code, not machine learning algorithms; and when they do, such as with "do I need to get over now to make the next turn", it's still fed back into the "business logic" that decides what to do and thus logged and audited when it's sent back to HQ.
As I stated elsewhere, these are physical safety-critical systems where the totality of the systems safety components cannot be expressed in software alone.
Remote vehicle telemetry is valuable of course, but as a tool to serve the validation process... not the validation process itself.
Vehicle telemetry cannot be a complete accounting of all of the interacting systems safety components involved here.
For that, like all other safety-critical systems, one needs exhaustive, controlled and physical validation.
So if I said yes, what's your response? No, of course.
Not saying everyone thinks it's a scam here, just that your argument it's not doesn't do much with the bias of already paying...
My argument is that since I’m paying month by month I clearly know what I’m buying and have properly evaluated the value proposition. The classic argument that FSD is a scam because consumers don’t understand what they’re buying falls apart when FSD is a no contract subscription service.
Sounds like FSD doesn’t actually FSD.
In other words, you know what you’re buying is not what’s advertised so it isn’t a scam?
I have learned that there is no point trying to tell a Tesla fanboy what the problem is. They will not listen. In the nytimes article, a guy bought another Tesla right after the previous one was totalled because of an FSD error.
The thing I have been telling myself is that fortunately I am not risking my own life for that.
The “totaled car” was a 2015 Model S which would have been running MobilEye software, not FSD. Stirring up hysteria is a classic tactic that news sites use to generate revenue. Always look at the title of the page. When you see something like “New York Time Magazine” there’s a good chance what you’re reading is not actually a reputable NYT article.
When you see something like “New York Time Magazine” there’s a good chance what you’re reading is not actually a reputable NYT article.
I guess you might not be getting scammed if you know it is unfit for any purpose, but at the very least you are being a danger to society for using a explicitly unsafe product that can not be operated safely.
I suppose what you’re saying is true, in the same sense that using any modern computer technology which has software in it’s stack, which hasn’t been formally verified, is fundamentally unsafe.
If an IDE on your laptop malfunctions, you end up with a crash or some malformed data.
If self-driving fails, you end up barreling down the road in a 2 tonne deathbox, likely killing you or someone else.
Not equivalent in the slightest.
By your own argument all adaptive cruise control is unsafe. Maybe you think the manual and warnings given to the consumer qualify "defining any operational domain where it can operate safely", but it doesn't give an exhaustive list of "here's where you can safely operate this", it just says to not use it when it's unsafe to do so - it's always "drive safely and take control when necessary" because the scenarios are literally infinite.
I get enjoyment from my heroin too, so scam is the wrong word.
But a crime is commited in both cases
Which is sad because they’re still the best value for money you can get in the EV space. The only reason I haven’t bought one is because I hate touchscreens.
I use lane following, summoning (which saves me time), and the windshield wipers every day.
Does anyone remember will it wipe? https://youtu.be/0SSYFMtdJ5k
I’m also furious about the hype, and how much of my time it has personally wasted.
I sometimes have to turn off the auto wipe, when there is a lot of glare…
They did this because they were (and remain) massively behind the market leaders, and the only way to catch up was to start bulk data collection early with a system that wasn't ready.
As to the baseline rate, the baseline rate for police reported accidents in the US is 1 per ~500k miles and fatalitys is 1 per ~80M miles [1]. That is approximately 1 per 30 years of driving and 1 per 5,000 years of driving, respectively using the reported average of ~15,000 miles per year per driver.
As to the availability of FSD data, you can see my other comment:
https://news.ycombinator.com/item?id=36181877
[1] https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf
That's about 80B test miles per year at about a capital cost of $0.50 per mile.
For a company that, say, spends ~10% on R&D, it would require the company be about a trillion dollars, which only Apple, etc. have reached.
Tesla does some of this, but then releases fast to the public... So far it has had some concrete consequences when they pushed this testing responsibility onto unaware people.
> As FSD is a safety-critical product, the burden of proof is on Tesla to prove in a verifiable way to third partys that it is safe. Absent that, it is assumed to be unsafe. So, no, it is the weakest allegation possible and the default assumption.
And what you are talking about in this comment is quite different. Please don't move the goal post (or actually, sure, safety certification makes much more sense than verifiable safety).
Again, I wouldn't find these objectionable statements in casual conversation, even as someone who participates in the mess that is certification.
"It is up to Tesla to get safety certification (that doesn't exist yet)."
That is obviously nowhere near what you wrote:
> As FSD is a safety-critical product, the burden of proof is on Tesla to prove in a verifiable way to third partys that it is safe. Absent that, it is assumed to be unsafe.
How does that in any way relate to safety certification?
I used extra words because I was expressing the core principle of certification which is to demonstrate to a third party that you achieve some set of agreed upon requirements. I did this because there is no current specific safety certification for autonomous vehicles, so I must appeal to the general principle underlying all safety certifications which is that you must demonstrate "safety" whatever that happens to be and that a product is not assumed "safe" by default.
Manufacturers are all pursuing self-certification in the US because regulators have been unwilling to answer petitions either way or commit to firm guidelines.
https://driving.ca/auto-news/industry/the-tesla-model-y-just...
If you compare the Model Y to the Corolla it outsells it. If you compare the Model Y to every car that is built on a Toyota B chassis (Corolla chassis) it blows Tesla out of the water. The B chassis alone has 10 different models, 9 of which aren’t Corollas. In other words, is a “JPN taxi” model that looks like a Corolla and has the same parts a as a Corolla not a Corolla?
Unfortunately, neither consumers nor the market have punished them for it. So it's inevitable that they will continue, and that other companies will follow.
I think there are two misconceptions in this space:
1) People mostly only talk about US companies and self driving cars in the US. China actually has a lot of self driving cars as well. Mostly following strategies similar to Waymo and with some borrowing from Tesla. And China is nowhere near as burdened by trigger happy lawyers slowing everything down. It's an ideal test ground for self driving from a legal point of view. And they are well funded. And they are running circles around most of the US in terms of car manufacturing. In general, the rest of the world will need to be covered by self driving eventually. Forget Phoenix; that's easy and rather boring. Can self driving cars manage in Italy, Spain, or conquer the German autobahn? I'm not pessimistic. But it won't be next year.
2) Painting this as a black and white game. All or nothing. Waymo is leading the way here by focusing on where it can work and gradually expanding their abilities. They are not even trying to make it work everywhere; just where they need it to work to make money. It's expanding it's area of operation and as it does the area it operates in adapts to self driving cars rather than the other way around. And it undeniably moves people from A to B at this point, seemingly without major incidents. The one thing that makes self driving cars hard is having human drivers around. Easy solution: get them out of the equation. This technology is basically going to be safety statistics and cost driven. As it gets better (i.e. safer and cheaper), it gets easier to optimize roads for self driving and the problem as a whole gets easier to deal with. Meanwhile there are big market opportunities in personal transport, freight, containers, etc. where self driving makes a lot of sense. Those are already happening.
My prediction is that Tesla will do well where companies like Waymo do well in as well and that they will meet in the middle sooner rather than later. And being positioned as a safety feature, all Tesla needs to keep on doing is failing to cause a lot of fatalities and gradually keep on improving. The rest is just raw numbers. As soon as there are millions of cars driving semi autonomously most of the time failing to cause a lot of trouble, confidence increases and it gets harder to argue there is a problem. Eventually, people will let go of their steering wheels. Insurance companies and cost will incentivize them. Based on the numbers.
The fact that they’ve figured out how to make income from unfinished self driving software makes them, in my opinion, likely to succeed eventually. For everyone else self driving is a money pit. Tesla can continue working the problem indefinitely until it’s solved.