What would my estate sue you for, exactly? Negligence?

Because a negligence claim requires my estate to prove that you failed to exercise reasonable care.

If the AI is generally reliable, it's reasonable for you to trust it and take your hands off the wheel.

If the AI suddenly goes berserk, it would be hard to prove that you breached your duty of care, and are liable for negligence.

> it's reasonable for you to trust it and take your hands off the wheel.

Even though the self-driving system explicitly tells you to keep your hands on the wheel and yells at you if it doesn’t sense them often enough? I don’t think it’s an accident that all of the current systems do a lot of work to make sure you’re aware you’re driving even if the car is helping.

This is building the precedent that manufacturers know whether their systems can handle being unattended, and as soon as they don’t do this they’re liable.

In the firework example, you can still sue the person who fired the defective firework, because they should have known that unpredictable behavior was a possibility, and taken steps to prevent that from escalating into some damage.

Likewise, suing the person behind the wheel of the self-driving car for negligence should work. Because the "non-driving driver" had the ability to seize the wheel, and should have known that some boneheaded behaviors are a risk with these early AIs.

Does this change of there is no wheel for the passenger to seize?

If you're the owner of the car then I'd say it doesn't change - you chose to take that vehicle out on the road knowing it didn't have a way for you to control it. You can take the steering wheel off of your car with a wrench, you could build a computer to steer it for you, you're still primarily responsible as the owner.

The owner could then try to go after the company, but they're still ultimately responsible even if the company doesn't cover the cost for them

If you are in a vehicle where there is no ability for you to prevent it from hitting you, then the responsibility falls in the manufacturer of the driverless vehicle.

You are expected to be in control, or be able to be in control at all times. If there is such a possibility, anyways. If there isn't, that means the manufacturer is the one in control.

I mean, I'm not a judge, so we'll have to see, but it seems obvious to me that the existence of a wheel implies that sometimes you may need the wheel. And that if there is no wheel, it implies that the manufacturer of the car is saying that you won't need one and their own software is better in every situation.

So yes, I'd say that would change things. IMO.

> it's reasonable for you to trust it and take your hands off the wheel.

The user undoubtedly agreed to some T&Cs that mean taking their hands off the wheel even to push something on the touch screen is negligence:

From [1]:

Before enabling Autopilot, the driver first needs to agree to “keep your hands on the steering wheel at all times” and to always “maintain control and responsibility for your vehicle.” Subsequently, every time the driver engages Autopilot, they are shown a visual reminder to “keep your hands on the wheel."

[1]: https://www.tesla.com/support/autopilot

From what I've read there's a lot of open questions (think of microsoft's linux FUD in the 00's) where you might be wholly liable, the company could be wholly, or more likely some mix of the two, Mercedes is offering some level indemnification for that because there's not a lot of precedent in court to say one way or the other.

if I was looking at a car with self driving capabilities I'd definitely be looking at Mercedes over pretty much anyone because of that.