"I don't think there are many excuses to be made in these contexts. If you've ever known your mother, or had a sister or a girlfriend who was attractive (and the recipient of suggestive comments), you know exactly what's acceptable and what's not. Picture it's your girlfriend getting harassed."

Your argument is incomplete. I doubt most people have an attractive women in their lives, having either the role of mother, sister, or girlfriend, since such a minority of women are attractive, even if they might be the only women you notice every day.