The problem is that you don't know what the car is going to do. E.g. if you see a barrier you would mentally prepare to steer. But if you trust the car you don't prepare and then suddenly have to take over at the last moment when you realize the car is doing something wrong.

That means the autopilot doesn't help at all since you would have to basically drive manually all the time anyway.

Exactly - the only safe way to drive using autopilot assist is to keep your hands on the wheel and mime steering every corner... and at that point how is it assistance?

Maybe one solution would be to actually show you what the car is "thinking". E.g. it could project the outlines of what it's seeing and what it's planning to do on the windshield.

Aren't there regulations/requirements about what's allowed on the windshield?

I would think it'd be seen as a "distraction", those suction cup mounts for example are illegal in a lot of states.

In this case I'd agree that the cons outweigh the pros. Surprising Tesla can legally market the feature or even offer it to the general public in its current form.

EDIT: typo