Unless, of course, the manual drivers counteract by driving more dangerously. Why stop for a red traffic light if all other cars are driven by computer? Can you try and get an autonomous car to drive into a ditch by moving your car very close to it?
I think there will have to be laws governing such behavior ('harassment of an autonomous vehicle'), with video cameras in the autonomous vehicles providing the evidence.
That also is a reason that I think driverless cars will not work everywhere for quite a number of years. In the typical old city in Western Europe, an automated car will have to negotiate with pedestrians and cyclists for room. Once the pedestrians know that a car will brake if 'challenged', they will change their behavior. I know I would (I guess driving a self-driving car through the center of London or, better yet, Amsterdam in rush hour must be one of the hardest challenges)
Also, why would pedestrians or even cyclists care about one way streets? Even if they do, consider the case where streams of cyclists enter a one way street just in front of an automatic car. That could easily happen in Amsterdam (for an impression, see http://bicycledutch.wordpress.com/2013/03/28/amsterdam-bicyc... or http://www.nytimes.com/2013/06/21/world/europe/a-sea-of-bike...)
Did you understand segregated routing to mean separate streets?