I'm not even saying this is an established scientific claim, only that it would be really weird if the statistical differences between men and women were 100% outside of the brain, given how different the evolutionary pressures are.

Evolutionary psychology does have some problems :) It's easy to tell dazzling and believable stories in that frame, but much harder to prove them empirically.

I think some real science is being done in the field, but I agree that when people make EP arguments on the internet, you need to apply a lot of skepticism.