And same sex harassment is probably worse. Ever seen someone whip out his junk at a workplace? Happened 3 times in my life.

One of the primary differences is that a man rarely feels physically threatened in a situation. If I'm working at 8PM with a colleague and nobody is in the office, I'm probably not worried about my female colleague physically assaulting me. The reverse, however, is not true.