> I don't think we understand the human factors involved

Do we? Because human factors responsible for literally every traffic accident in history, and those don't seem as scary to you. Basically: it's fear. It's just fear. And it's reaching a peak right now (and being deliberately pushed, frankly) just as Tesla is reaching a public rollout.

And why? Because there's another fear at work here too. If this rolls out and doesn't kill people. If it turns out that it actually is safe, then a lot of people across a lot of industries stand to lose a lot of money. Stories like this are, fundamentally, just marketing press hits. The public FSD beta rollout in some sense marks the last chance to kill Tesla's autonomy product before... it wins.

Six months down the line they are using the FSD drive on a twisty section of road. The sun is the eyes of vision based FSD and there are a bunch of cyclists in the road. The FSD hasn't been trained on this case (or hits a bug or whatever) and doesn't detect the cyclists. The driver who has been "Trained" by the FSD to not pay attention is fucking around with their phone. The tesla plows into the cyclists.

This is my fear - the fear that you detrain people on the responsibility of driving. I suppose at the base of this is I have very low trust in how people will reason about their role with technology. People who don't know about the limitations of the machine will assume it is accurate. People will work to exploit the limitations (two hands on the wheel, eye tracking, etc).

If this technology was marked as "crash avoidance" technology I'd be much less worried. To take the twisty road example, imagine if the Tesla can not only self drive the car to a safe point in case of drive incapciation - that's pretty awesome but also puts the clear responsibility on the human that they are doing the driving.

Again, that's a multi-mode failure. You're assuming that (1) edge cases must exist somewhere where FSD is unsafe, (2) that drivers will find them, and (3) that when they do, the existing supervised beta paradigm won't work. In isolation any of those can happen, sure. But in combination they're vanishing.

Broadly you're doing the luddite/reactionary thing and demanding perfection: as long as you can conjure a scenario in your head where something "might" happen you want to act as if it "will" and demand that we refuse to implement new solutions.

But the goal isn't perfection and never has been, for the simple reason that the existing solution also sucks really bad. Real human drivers in Mendocino with the sun in their eyes hit bikers too!

Nope. If you test your software and it shows no bugs, I'd be more worried that the test was flawed. Underneath the shiny bits we developers know how buggy software really is. Its just a fact that the smartest humans keep introducing bugs in software. The more complex the software the more complex the bugs and difficult to find.

> Stories like this are, fundamentally, just marketing press hits. The public FSD beta rollout in some sense marks the last chance to kill Tesla's autonomy product before... it wins.

Nobody owes it to Tesla to give them free publicity or be forced to immediately accept their ideas. Personally, I think its makes more sense to invest in tech that reduces dependence on needing cars in the first place.

Don't these people regularly have to intervene and take over, hence the no accidents? Unless I can responsibly cede full control to the self driving system it's less than useful for most drivers imo. If I'm wrong let me know but that is the impression I have taken away reading on the tesla self drive system.

And there's no evidence of a safety problem.

At the end of the day it just needs to be better than the average human, and that's frankly not the highest bar unfortunately.

This comes up a lot on techno-centric forums like HN but I think it misses an important point. It’s not necessarily if it performs better than the average human that will lead to policy that allows it on roads. It has to have a high degree of trust in the public’s eye. That bar is likely much higher than “just better than the average human.”

Arguing that full autonomy isn't possible yet (duh, I mean, it says "beta" right there in the name) isn't an argument to disallow testing.

What are you gonna do when a car goes rogue and murders somebody?

Do you feel the same way about your gas furnace or the autopilot of the last airliner you flew on?

According to a Rand corp. study I quote in another comment, it will take millions or billions of miles and decades or centuries before we can know self-driving cars are safe, simply from observation:

Autonomous vehicles would have to be driven hundreds of millions of miles and sometimes hundreds of billions of miles to demonstrate their reliability in terms of fatalities and injuries.

Under even aggressive testing assumptions, existing fleets would take tens and sometimes hundreds of years to drive these miles — an impossible proposition if the aim is to demonstrate their performance prior to releasing them on the roads for consumer use.

https://www.rand.org/pubs/research_reports/RR1478.html

That's statistics, btw, not fear. Data, like you say, will "win" at some point but that will be far in the future. Until then we have a few thousand systems whose safety we can't measure but are available commercially (and sold under pretense of improving safety).

Note also that before Tesla started selling cars with "autopilot" there was no way to know how safe it would be, in advance. That is not the behaviour of a company that cares at all about safety, other than as a marketing term.

The goal isn't certainty. It's "better". It seems like it might be "better" already.

Is there some other published work that you prefer over the Rand study?

And it's spun anyway, since most of those "trillions of miles!" numbers reflect things like proving 95% confidence of 100%+ improvements in safety. When all we really want to know to release this to the public is 95% confidence of 0% improvement (i.e. "is it not worse?").

Are you saying one cannot use statistics to make predictions about future events?

The idea of even the most modern intelligence system navigating through an environment as elaborate and dynamic as New York City or downtown Chicago should give anyone pause. I can see them being more useful for long distance journeys such as hauling a trailer across rural desert areas where all the vehicle needs to do is stay in its own lane, maintain a set speed and brake if something obvious gets in front of it.

Perhaps it's not the best idea to use these more advanced Level 4 systems in the densest city yet, but most of driving does usually happen on highways and regional roads so it's not a major loss.

I'm sure someone will eventually figure out how to detect "eyyy im walkin eer" and honk back automatically, and then it should be usable in NY as well.

> versions of AI that are hidden from the public both private and military

I do think you overestimate how much the military has its shit together. These systems need gigantic amounts of driving data to train and nobody else has that aside from car manufacturers with their fleets. And out of those Tesla is the one that's been doing it the longest.

That said, personally, I think it's unlikely.

First, the vast majority of AI research is happening out in the open. It's often funded by militaries (e.g. DARPA) but it's not owned by them directly, rather the owners are universities and similar.

Second, private corps promote their AI advances very aggressively and this marketing is a big part of the advantage they try to establish over their competitors, so it doesn't seem obvious why they would keep a new advance "under wraps". For example, what would Google do with a secret machine vision system capable of 20% better performance than that of its rivals? I can't tell.

Of course we can assume that Google or someome similar have a much more advanced system than anyone else, like a real AGI. We can imagine such a system sitting at the heart of Google, dirctly advice Larry and Sergey how to run the company. But then, if we can imagine that, we can imagine anything.

And I do mean anything:

https://gatherer.wizards.com/Pages/Card/Details.aspx?multive...

As in security cameras, finances, cell tower data, etc. Facial and gait recognition, location tracking, etc. far beyond anything commercially available. But I'd doubt they have an AGI behind closed doors :P