You're being downvoted, but you're not wrong about the strategic risk. I worked on that for a while and at least got people to take it seriously. When I started there weren't even regulations coming for self-driving cars. At the end of the day, we use nukes to deter against strategic risks and if undeterrable adversaries have strategic capabilities then we're doing it wrong.

“My personal guess is that we’ll achieve Full Self-Driving this year at a safety level significantly greater than a person."

The preceding bit, just as bad IMO.

I'm expecting a Ghost in the Shell type attack on Tesla's update software that gives all the Tesla's self driving, and they auto-drive themselves to the next protest, ala Freedom Convoy without the truckers. Then brick them, once they reach the protest, so they can't be driven home (only towed).

Airplanes were very dangerous at first, but here we are today.

General Aviation is in totally different situation than what Tesla is doing. GA is so over-regulated that we're flying decades old small planes as it is financially not feasible to bring something new to market and have it certified. Progress is mostly done on experimental/ultra-light planes. Those suffers similar issue as they are limited to certain weight and even feasible safety features often can't be added due to weight limits. On the other side of safety spectrum you have Tesla that is pushing something-like-autonomous-driving software to common people to try out on real streets.

The majority of those aviation rules and regulations are written in blood. While unfortunate, it's not surprising to me that automated vehicles may follow the same path. Regulators are rarely forward thinking.

The problem isn't that regulators cannot be proactive.

The problem is that no one wants to be regulated. Just look at crypto, people love the lack of regulation. However, once there is a crash or enough people get scammed, there will be demand for regulations to prevent that from happening again.

Weight limits, and then electrics. A lot of GA planes have multiple things - lights + radios, wired into the same circuit because there's not enough circuits to keep them all separate.

This is a very ironic statement to be making given the subject and the recent affairs with Boeings software updates.

If cars had the same death rate per mile as airplanes, tens of thousands of lives would be saved every year in the US alone.

To first order, planes only crash when they're landing or taking off.

Cars crash in pretty much every situation. Fatal incidents are more clumped but no where near as much as planes.

I vaguely recall that the risks are closer than you'd expect when going by hours of flight, rather than distance. I'm not finding the numbers right now though.

In https://www.pbs.org/wgbh/nova/planecrash/risky.html I did find some older numbers. For commercial carries, it was 1.9 deaths per vehicle mile instead of 1.3.

https://www.pbs.org/wgbh/nova/planecrash/risk-01.html (99 through 03)

However, private planes are more risky and don't seem to be included in those numbers.

Pithey comment that is true based on current usage patterns, but a total lie when everyone starts flying to work or the grocery store... and they're the pilot.

Agreed, the human pilot is the problem. The sooner we can remove them from the situation, the more lives will be saved

If drivers got the same training and stuck to the same rules as airline pilots, we'd have a lot less accidents. Most accidents [citation needed] are still caused by reckless driving; inattentiveness (exacerbated by driving assistance), speeding, drunk driving, aggressive driving, etc.

Moving to automatic driving will probably be a thing long term, but for the short term, it's a human problem. If only people were held more accountable for driving badly.

Don't worry, that will be enabled at a later date. Mid-Atlantic.

Even the 737 MAX was very safe though. That's why it was notable: it was a genuine safety regression with a clear root cause in an industry that had been getting steadily and inexorably safer for decades and decades.

Sure. It's a plane that never crashed in the entire Western Hemisphere and Europe. Among "unsafe" things that's pretty good.

What's so unbelievable about it? It's on brand

https://video.hardlimit.com/videos/watch/ede3d957-2a1f-4ec4-...

I don't think critics realize how much testing has gone into this technology

Not that much, honestly. They've already switched tech stack multiple times. They recently removed radar from new cars, the 'camera only' stack hasn't been tested that much. They also have mostly tested on and built for California roads and weather.

My Tesla once tried to steer me into a guard rail (multiple times) because the setting sun caused a lens flare. You can't tell me that this is well tested technology that will enable level 3. Only after doing that twice, autopilot noticed it and disables autosteer.

The cameras also have condensation and freezing issues, causing them to not work on cold, wet days.

This was a car that was, according to Elon, fully equipped for self driving.

I really enjoyed autosteer, but there's major flaws in the current implementation and I seriously doubt they'll deliver on current hardware. If they don't, I'm really curious about the outcome of the lawsuits.

The nature of the business is that artificial intelligence of all flavors is extremely poor at being 100% correct. 90%, maybe one day, but not now.

The concept that automobiles will travel without human assistance to override IO mistakes from sensors is ridiculous.

I don't think fanboys realise what Elon is, what's his strategy, what self driving requires, the state of the current tech, &c.

So far it's ridiculously underwhelming, even I didn't think it was going to flop that bad and I was very skeptical to begin with.

Tesla users are experiencing some sort of Stockholm's syndrome, it's scary

Billionaire worship. I really don't get it.

I wonder. Even with the (loud) minority of extreme fanboys, I'd argue that most fans probably know more about Tesla's strategy and the current tech than most others.

Another, then. These fans are often the ones parroting the entirely incorrect "You have to have incidents and near misses in order to train it to do the right thing next time".

I've heard that way too often.

I think proponents underestimate how much testing is needed to make such a system actually safe.

edit: FSD is clearly still alpha; ive seen it steer towards on coming traffic, and almost cut off large vehicles but for human intervention.

I think online commentators don't understand how low the general public's bar for "safe" is.

People will be elated if they can buy a car that can chauffeur them around as well as a teenager who's had their learner's permit for two days. At least with the car you're already in the driver's seat and don't need to articulate to it what it's doing wrong. Sure you'll have to keep an eye on it to make sure it's not setting itself up for a situation it can't handle but you have to keep an eye on the teenager too.

Crappy half-baked automation is prevalent the world over in all sorts of situations. The normal, unwashed, much derided masses are perfectly capable of learning its quirks so they can keep it out of trouble. The idea that a car "can't see" bollards or bike lane marking or partial lane obstructions or whatever is not a hard one for people to grasp.

The solution to "teenagers having their learner's permit for two days are dangerous" is not to put an untested, shitty OpenCV car on the road, it's to make your driving tests and lessons more than just "drive twice around the block". I've seen Teslas do things learners would never do.

and who's going to be responsible when that car drives full speed into a mother and her 3 kids?

Why don't you plug that situation into my analogy and see what happens? The existing legal framework is already set up to handle this.

Considering how strongly HN advocates for strict liability for car drivers I find your reply baffling. Why wouldn't the driver be liable?

What driver? It's self driving. Companies putting machines on the road that murder innocent children will surely be held liable.

Pieces of software like OS kernels and browsers run code quadrillions upon quadrillions of times per day and yet still suffer hundreds, if not thousands of critical security vulnerabilities. Testing only demonstrates the presence of bugs, not the absence.

There's a whole field of high-assurance software, with verification, formally-proven compilers with translation validation. Only there is there hope that one can write software and prove it, with a machine-checked proof, correct. Tesla is not that far. Not even close.

But no matter, let's just deploy it anyway.

I think the critics see the amount of bugs in code that has been tested much more than self-driving cars and have very reasonable doubts about giving over something as important as driving. Even without (say) foreign actors trying to hack the cars or jamming the sensors, they seem to keep driving into stationary objects all on their own.

It’s not good enough to market as self driving. It’s a fancy cruise control with lane assist.

The whole thing is a stock pump that went too far

If you're in the beta program or have access to youtube, you would know that is not what it is at all.

Videos like these ? https://www.reddit.com/r/IdiotsInCars/comments/rgiu8m/the_fu...

Wow, you cherry-picked 10 seconds of video from 1000s of hours of content! Amazing work! That video was 4 minor versions ago also.

The thing is when you want to go global with a "10s in 1000 hours" problems is that it'll happen every single day to thousands of people. It's all fun and games when you're a website or a mobile app, not so much when you're a 500+hp two tonnes metal cage using public infrastructure as a beta test sandbox

Was that the minor version that introduced phantom braking? Or the minor version tossed over the fence a few days later that "fixed" phantom braking? Or the minor version after that re-introduced a different phantom braking issue? Or the minor version after that...

Yes, if you have access to YouTube you'd know it's not even a functional lane assist.

Care to elaborate, for those of us who are perhaps more ignorant on the subject?

Sure, the OP said that Tesla FSD is "fancy cruise control with lane assist".

If you have FSD Beta 10.10 installed in your car (which 60K+ people have in the US), you can input a destination and it will take you there. That includes stop/go at green lights, changing lanes, going around parked cars, making un-protective left turns etc.

It's not perfect, but it's beta, and the iterative improvements have been very noticeable over the past few months.

Critics mostly have made the correct observation that the investment, time and effort required for each increment of improvement seems to place the problem in the “theoretically possibly but probably practically unreachable” domain.

Tesla can’t even implement auto breaking on a straightaway that doesn’t kill model pedestrians in controlled testing. Only the truly credulous think they can get to FSD in any realistic timeframe let alone Musk’s fantasies.

I don't think Musk zealots realize how stupid it was to demand it work with only camera sight, banning LIDAR, so the AI has piss data without depth perception to work with

Tesla's FSD will never come to fruition as a stand-alone system.

Funny thing is, they removed Radar because it provided less accurate data on depth than their vision system. I doubt that there's much gain in including an additional source of depth information.

Sure, it might be more precise than what is possible with vision alone, but it's also not needed.

Source: Musk said so and provided no proof.

It is the absolute standard in any safety-critical industry to have multiple sensors all feeding data and making up for each other's weaknesses. Cars are not excluded from that.

I refuse to believe that Tesla's image-based AI can discern better distance and depth information than LIDAR (even if they're using cameras with calibrated positioning for stereoscopic vision or something) without better sources than "Elon said so."

And I am talking about LIDAR not RADAR, even though RADAR also could be used as yet another sensor in the stack. Using only regular vision is a typical dumb Musk thing; LIDAR was quite a bit more expensive when these self-driving things begun to take hold and Musk didn't want that cost so he went all-in with his own dumb solution of "software will fix it."

Reminds me a bit about how Musk originally wanted to fly people to Mars, by 2014, with a launch escape system on the rocket.

By now the launch escape system has been scrapped wholesale, the new approach; Just have to fly enough rockets, without exploding, until safety becomes a side-effect of economies of scale and improved tech.

It's a combination of "tech your way out of the problem" with "grow your way out of the problem". Which might be a viable approach for creating non-integral software products, but seems like a really bad fit when peoples lives are at stake.

No one "banned" LIDAR.. I think whatever comes to be will rely on more than just vision at some point in years to come. Regardless of what Elon says.

I agree it will need to go beyond cameras if it ever works. But the question becomes, will Tesla be on the hook to upgrade the sensors in cars that are camera only, or will they instead deploy unworkable software to those cars?

That could be true, but given that few systems cannot be compromised by a determined attacker with resources, it seems worth thinking through what happens if a fleet of these is controlled remotely by people with bad intent.

Compared to not hotdog app? Yeah, a lot.

Compared to actual safety critical automation systems, designed to operate with humans? It hasn’t even started.

Edit, as I pressed enter by mistake.

What’s happening now isn’t testing, as a system as whole isn’t ready for that. It starts with proper design and long validation, before you can start talking about proper testing. Tesla is rewriting all parts of it constantly. What’s happening right now is mostly PR exercise, plus collection of low quality training data.

I don't think defenders realize how much more testing is needed for this technology.