Where I live, bicyclists should by law be either on the road or on a bicycle path or one of a few designated shared bicycle/footpaths. There are really too many pedestrians for it to make sense to use the sidewalk for bicycles inside the cities, and of course the benefits of commuting by bicycle disappears when you have to move at the same pace as pedestrians.

Also, from my experience with traffic in general, it doesn't take many assholes to make a situation uncomfortable, nor does it take an asshole to be inconsiderate. If one in ten road users or pedestrians I meet every day has a sudden lapse in judgement, that's enough for it to feel uncomfortable.