> any man who claims he's a woman can, with impunity, impose himself upon spaces that were only ever intended for women and girls

The military policy (that I assume you're defending) is to ban anyone with a "history of gender dysphoria". Your points would have a little more weight if the situation was like in 2018 when individuals could serve under the condition of being stable for 18 months in their identified or assigned gender. But not much more weight, since that is still a lot weaker than what you describe.