Humans are notoriously bad at just paying attention and not being in charge. Those few seconds their actual attention is needed are critical.

I do appreciate that my car can do full distance control and assist if I am drifting, but it doesn't control itself, so I can never disengage. Personally I feel that this is wonderful and should be the limit. Anything past that should just be fully autonomous.

100%. If I'm driving and my attention wanders, I need the immediate feedback of drifting out of the lane and having to abruptly course correct, and being like "dang yeah, let's not do that again", possibly reinforced by passengers scowling at me from elsewhere in the vehicle.

I can't imagine trying to focus on supervising an AI pilot without that kind of feedback.

interestingly, I don't have much trouble with that, as it works exactly like most airplanes that I fly as a pilot.

An airplane autopilot is a dumb device, in that it does execute _exactly_ the plan you tell it to, and it is up to the pilot to at all times decide whether the current plan still makes sense or needs to be altered. So the pilot makes the strategic decisions, and leaves most of the physical tasks of flying to the autopilot.

I find myself using my M3 w/FSD in exactly the same way, as that I put on autosteer pretty much immediately when I'm out of the driveway, but I constantly nudge it into the lane that I want it to be in (by using the turn signal) or push the accelerator when I think it is taking too long pondering a turn. So i leave the physical driving (keep lane and distance) to the car but manage the car to always go exactly where I want it.

I have no trouble staying alert this way when doing medium long drives. Long highway drives where autopilot is so good that it requires no manual interaction is where the trouble starts and I find it hard to keep paying attention.

This is where in an airplane you have a copilot and can discuss strategic things like overnight stops, fuel stops, etc... Maybe Tesla needs a built-in chatbot to make me do that :)

Isn't airplane autopilot considered to be significantly easier than automobile autopilot? Fundamentally, it's PID to track a bearing and elevation, and the worst kind of emergency it's likely to encounter is rough weather or some kind of mechanical problem with the plane itself, neither of which need to be handled with sub-second response time the way many road emergencies do.

I get what you're saying about the physical/strategic split, but my perception is that automobile autopilots are simply not good enough to be trusted with the "physical" stuff in a hands-off way like an airplane's can. And that's mostly because driving a car in a straight line down a highway is way harder than flying a plane in a straight line in an empty sky.

"I have no trouble staying alert this way when doing medium long drives. Long highway drives where autopilot is so good that it requires no manual interaction is where the trouble starts and I find it hard to keep paying attention."

Exactly!!!!!!

The trouble is when your attention wanes and you don't know exactly when you should be paying attention. What I found is the small amount of automation my car does is wonderful to keep my exhaustion down. When I drove for 4 hours in my 2012 prius, it was a chore, i had to do ALL the mental math myself. When I drive my 2023 CX-50 I have to pay attention at least 30% less, and that 30% is a massive difference, it feels like 90%. And any time my attention wanes, the car starts complaining because I am drifting or not putting enough immediate response to the road. It becomes a "pay attention quickly" wake-up call that happens within seconds of attention waning.

The worry is when the car doesn't snap you back into that attention mode, and you just trust it, right up to the problem.

I suspect the below-"Level 5" driving systems will become more of an "augmented driving". I've driven in newer vehicles with automatic lane centering, pedestrian detection, etc. and they don't really seem like they're even doing anything, you still feel like you're the one driving, except that it's more precise with the occasional interruption by the car when it perceives risk of a collision.

These augmented systems will probably reduce the risk of accidents so greatly that the value proposition for Level 5 driving systems just won't be there.

> These augmented systems will probably reduce the risk of accidents so greatly that the value proposition for Level 5 driving systems just won't be there.

I've driven a lot for decades and frankly enjoy driving. I drove from TPA to SLC via PHX and back for fun. But I will pay $500/mo level-5 subscription for a comfortable car that drives itself.

I'm trying to figure out how many hours a month are you in a vehicle where that makes sense for you?

$500 seems reasonable compared to loan payments, full insurance, and parking. The math may not make as much sense if you would otherwise own your car outright, though.

Having level 5 available honestly opens up tons of options that were not available before. A 90 minute commute is so much more palatable when you can be sleeping. So are road-trips.

I'm not sure making 90 minute commutes more palatable is going to be a good thing overall, unless we somehow incentivize reduced travel in other ways. Better distribution of amenities and logistical improvements, maybe... More distributed work environments, perhaps coworking spaces, but with a less-stupid business model.

That's $17/day. If it gives you an extra hour a day and eliminates driving-based stress then it seems very well priced. Especially if it includes the car

I'll also add that I think what Waymo is doing right now is closer to a semi-autonomous streetcar. There's probably immense value in that approach, especially as an alternative to mass-transit systems that have costly labor, but it's not clear that they are imminently close to "anywhere, anytime" self-driving.

"Augmented driving" is Level 2. That's where commercial products (GM, Mercedes, Tesla) are now.

Volvo was talking about level 3 back in 2017, but they gave up.[1] Level 3 means that the system may ask the driver to take over, but if the driver does not do so, the system must get the vehicle to a safe condition. Preferably pulled over out of traffic, but as least stopped without hitting anything. The driver is not required to watch the road.

The serious players are trying to get to level 4, where the driver is not expected to take over but the set of roads you can use is limited.

[1] https://www.youtube.com/watch?v=2q00jIBhkq4

The leveling system is a bit off. Level 3 doesn't mean "better" than Level 2. A Level 2 system might actually offer the best safety profile of any of the Levels. That's what I'm getting at: there's a lot of runway in Level 2 systems, and I think they'll be so good that it will kill momentum for Level 3+ systems.

The problem is that when a level 2 system gets too smart it can confuse drivers and lead to bad reactions in response.

e.g. let's say you have a level 2 system which starts auto-evading, suddenly steering without user input, the user is likely to reflexively try counter-steering in response.

Yes. Eliminating or vastly reducing the head-on collisions caused by drivers drifting across the center line, and the rear-end collisions where they don't see the stopped or slow car ahead of them, are a huge win. I'd trade full self-driving for really effective lane departure warnings and auto-braking collision avoidance any day of the week. Next step (or included) would be reacting to red lights/stop signs if it appears that the driver is not stopping. Deal with those things well and you've eliminated the causes of most serious accidents.

I think Mercedes has level 4 on highways now? I think this is the way forward actually, let cars drive cars themselves on the long boring bits (which are actually easy for AI) and leave the driving to the humans everywhere else. Having tried many augmented systems I don't believe in self driving in varied conditions within 10 years. I think the locations where waymo operates is a good indication of what is possible at the moment.

> These augmented systems will probably reduce the risk of accidents so greatly that the value proposition for Level 5 driving systems just won't be there.

The value proposition of L5 systems is also the not driving part.

Here's Chris Urmson talking about this 7 years ago when he was still working on (what is now) Waymo: https://youtu.be/tiwVMrTLUWg?t=169

Aren't they geofenced, which would make them level 4?

>Anything in the middle (like what Tesla is doing) is IMO more dangerous than useful

Intuitively I would have agreed with you, except Tesla has been doing it for years and their cars are statistically safer by every metric (fatalities, indicents, etc.).

> their cars are statistically safer by every metric

Compared to what population? Older, tech-savvy people buying 60k-120k usd vehicles? And do you mean in full self-driving mode Beta, which Tesla won't allow you to use unless you have a track record of save driving?

> Anything in the middle (like what Tesla is doing) is IMO more dangerous than useful.

Even if a hypothetical, future Tesla FSD sometimes crashes in ways that could be prevented had the driver payed attention, it could still be statistically safer than a fully human driver (ie the number of FSD crashes even if left unattended < the number of crashes by humans driving).

To clarify, I'm not talking about the current state of FSD, I'm talking about a hypothetical, future Level 3.

Doing such aggregations is pointless. The majority of traffic accidents are caused by drunk drivers, drivers who are too young/too old, people on their phones or otherwise distracted, people driving in bad conditions, people driving unsafe cars etc. So yes, while Tesla autopilot may be better than all of them on average, I will still only use it if it is better than ME.

> I will still only use it if it is better than ME.

You're free to not use it. But if Tesla FSD is safer than the average driver (even if that's because the average driver is on their phone) then, going back to your statement, it is more useful than dangerous for the average driver.

What is your definition of "average driver"? There are ~250M drivers in the US and a little over 2 million car accidents that result in injuries every year. So the average driver is doing perfectly fine. In fact 99% of them are fine. Car accidents don't follow a bell curve. If you want to make the roads safer you need to target the bottom 1% of drivers, but there is no guarantee that they are the ones who will adopt autopilot. If everyone else takes their eyes off the road it will actually make things worse.

What are your thoughts on airplane autopilots then?