I can see that there will be a rapid change in social attitudes, once automated driving reaches the point where it is generally accepted to kill/maim fewer people than automated driving.

Think back 30 years, when it was socially acceptable to light up cigarettes around small children. Now it is socially unacceptable to do so, to the point that it is illegal in many jurisdictions. Where I live, it is illegal to smoke in the vicinity of a school, within 10 metres of a playground or in a car with a child under 16 years [1].

Once a tipping point is reached, where manual driving is widely accepted to be more dangerous that automated driving, manual driving will quickly be banned in the vicinity of schools. There will be resistance and arguments from manual driving aficionados. Eventually, public manual driving will be confined to either remote areas or purpose built facilities, just as public smoking is restricted to open-areas and smoking rooms.

[1] http://en.wikipedia.org/wiki/List_of_smoking_bans_in_Austral...

Edit: On reading other comments, I'd extend the analogy to smoking (and other activities that carry a public risk) by pointing out that there will be a financial penalty for choosing to drive oneself. Just as smoking is a risk factor that increases health/life insurance premiums, manual driving will also be a factor in calculating insurance premiums.

Most manufacturers don't even produce manual transmissions on the majority of their models in the U.S. any longer, and, in the models that do have them available, they're sometimes more expensive. More than likely, the manual driving controls would become something like the pseudo-manual shifting available on some automatic transmission vehicles: you get a little more control over how the vehicle operates, but you won't be permitted to do anything that will be likely to damage the vehicle. I think we're already seeing a lot of this in the safety features of some higher-end cars (collision avoidance/warning systems) and features like automatic parallel parking.