1. As with people who row or sail, just for the enjoyment. Such usage could be restricted to specific places like racetracks.
2. Because I'm going somewhere I "shouldn't" and I don't want to tell my car (and have it record my journey for someone else to review, or refuse to take me). This may also be the case with emergency vehicles, an ambulance may need to drive along a pedestrian-only street, for example.
Now, there will be a need to mix human vs autonomous traffic on the run-up to that. Obviously, until sufficiently many cars are autonomous to warrant the removal of human-controlled cars, that will need to be the case. Furthermore, early production Autonomous cars may have routefinding flaws, or the computer may break down, meaning that someone may need to drive a normally autonomous car themselves.
Personally that's already how I cross the street, and my confidence in human drivers isn't anywhere near so great.
I could easily see a bunch of kids deciding they'll go challenge the AI by dancing in the middle of a crowded street. Admittedly a millionth chance occurrence though. "Computer, take a picture, get me facial recognition, and report them to their parents."
Not perfect by a long shot, but unless automatic vehicles offer some similar pedestrian signaling capability, there will be issues.
There's also the question of what happens, say, when a vehicle is surrounded by a crowd of people with ill intent. This happens:
http://www.nbcnewyork.com/news/local/Motorcycle-Gang-Attack-...
Working on it. http://www.technologyreview.com/view/427743/how-do-you-know-...
The future, at least in terms of the dangers of driving, is bright.
I look forward to the day when every car is a taxi, and the only cars parked on the side of the road are the ones owned by the very few people who can still be bothered at all to own a car full-time, which will hopefully be very few people.