It surely is. People actually have trouble framing anything as sexist against men. All the men in a hostage holdup can get shot, and someone will talk about how it's sexist against women.

When no woman is involved at all, it's often called homophobia. But the hint there is that if no LGBT person or LGBT attribute is involved in any way, it's not homophobia either. It's just sexism. Against men.