Sure you would pay more than owner of automated car, but probably not more than you are already paying. People on bicycles don't pay any insurance at all, I don't see anybody complaining about them (for that reason anyway).
I don't think those are impossible problems to solve. For the leisurely ride, define a radius or area and let the car do a 'random' walk (or ride on only small roads), or visit beautiful spots where g+ users have shared pictures, etc. For a place with no address, give a lng,lat or drop a pin on a map.
I love to drive. I do not enjoy dealing with major backups. When I'm leisure driving, it's out to a small town in the middle of nowhere later in the afternoon or when everyone is at work; or, it's out in the boonies because wow that view when I take a certain turn is just absolutely spectacular.
Automated driving is only useful if you know where you're going. If you're on a leisurely ride through the country or are looking for a place with no address, you can't use automated navigation. So disabling it would be a necessity for a variety of driving purposes. As an aside, motorsport enthusiasts would be up in arms if they had to pay a premium just to be able to drive their expensive sports sedan the way they want on a closed track.
What will probably happen is a black box will record if the car was in automated mode when a crash occurs. If it was driving itself, the manufacturer will probably be at fault and forced to pay the insurance company. If you were driving it, the standard rules apply.
I have to say though, automated drifting is really depressing. I used to drive a Toyota MR2, which is an ass-heavy car that was very susceptible to snap oversteer. Unlike most other cars, if you felt like the car's rear end was about to slide out of control, you had to PUNCH the gas to get traction back. You basically have to fight all your instincts and do something crazy in a split second or end up spinning backwards towards god knows what. It's the most exciting thing i've ever experienced in a car (even more than sitting in on hot laps with an indy driver in a Lexus IS-F), and I would never get to experience it with automated driving.
As for non-drifting, automated driving is definitely an amazing and useful feature. Writing code on my way to work sounds like fun. And I know it sounds controversial, but if there was some way to couple a breathalyzer with automated driving, we might be able to use our cars as taxis that pick us up and drop us off without us having the ability to drive. But perhaps that's hoping for too much...