The issue is that men wearing skirts is referred to as "crossdressing". When really it should just be men wearing what they like. If a man wears a skirt, then it's a man wearing a skirt.

That's the real way forward. "crossdressing" has a stigma because the very meaning of that word comes from gendering clothing. So you need to stop using that term.