My understanding is sexual harassment is aggressive, in a way that goes beyond confidence or assertiveness. I suppose it feels like being in a fight in a way. Most fights don't have lasting consequences, most fights occur in environments or cultures where they're almost excused in a way, but if somebody wants to fight you, and even if somebody wants to fight you in an environment that's normally very sober, such as your office, you tend to know it. (And depending on your relative sizes and prior levels of acquaintance, it will make you uneasy).