Fair question. I would say the primary difference is that in most workplaces men are seen as being competent, and being male is not a disadvantage or put you in a minority. There are undoubtedly professions where stereotypes about men would be harmful. For example if a man wants to become a flight attendant, nurse, or a school teacher.

Also consider examples outside of work, like dating. The tall, handsome, muscular guy is more likely to be perceived as desirable even though those traits don't necessarily correlate with what makes a good boyfriend. A man in search of a partner might feel objectified if his potential partner had posters of good-looking men up on her wall.

Actually, it turns out that even in woman-dominated professions, being a man is advantageous. [1]

[1] The Glass Escalator: Hidden Advantages for Men in the "Female" Professions https://www.jstor.org/stable/3096961

This is not about personal "pleasure in the physical appearance", but about the culture where using Playboy photos in a professional cobtext doesn't raise any eyebrows.

  where these hot guys throw off their shirts at the hint of an opportunity ?

I don't know why, but I'm reminded of https://www.youtube.com/watch?v=spwTDpJ0-eY