The risk is that some go-getter will use a harassment thread as leverage over you. No matter how saintly are you, in the current culture nobody is going to believe you if you're a man accused of harassment - so why open yourself up for the possibility?