I think there's a difference between initial awkwardness or fear of rejection -- which are totally valid feelings in social interaction, and not just limited to men -- and fear of being accused of harassment for making polite conversation.

I also think that it's easy to get sucked into circle-jerks that promote and exaggerate these fears of being accused of harassment. As long as you're considerate and respectful, no reasonable person would think you're harassing them. There are always unreasonable people, that shouldn't divide us into men vs women, into groups that fear interacting with each other.