This was true for me, but now days it almost never happen. What changed was that my route got a serious upgrade, with lower speed intersections, better yield signs for everyone. All it took was a well thought out plan for all the streets with bicycle traffic.
But yeah when I am not on my daily commute you still get that kind of behaviour.
Some weeks it can be really bad, several a day, some weeks it will be almost silent (but always at least one nutter).
The weather makes a big difference, for some reason drivers drive a lot worse in the rain and really hot weather, usually speeding / running reds and not checking mirrors more often than in overcast / typical UK weather.