But other characteristics such as nutrition and genetics matter more: the average German woman (5’5”) is significantly taller than the average Peruvian man (5’1”).
My point here is that the difference between men and women on most metrics tends to be more heavily influenced by other external factors.
I think that's the problem part of your argument. There's no qualification of what "most metrics" includes, much less whether that holds true for must neurological effects (which is more to the point), and that's before we even get around to discussing the fact that I'm sure there's a bunch of metrics we don't even know about.
Your statement might be entirely true. Unfortunately I don't know enough to judge the claims in it, I don't even know if we're at a level where we can judge the claims, and regardless you didn't provide any evidence to support it or help with that in any way. I'm not sure how it brings us any closer to an understanding of the issue.
Edit: rationale on down votes would be appreciated...
Given the topic, I took that as an argument that the neurological differences are not as pronounced as what is created by external factors, and I was responding to the lack of evidence provided for that view (which I interpret as the opposite of the view you suggested).
I actually lean towards your position, but I wasn't trying to express that as an argument (I'm unqualified to). I was just trying to point out what I saw as at best very loose reasoning (at least when provided without evidence).
Funny enough, estrogen has also been linked to increased aggression just as testosterone. You take a baboon and look at when females act most violently and it spikes just as estrogen spikes. Same for violence in women prisons. At the same time, depression also spikes which further suggest that estrogen causes wild mood swings.
But when we look at more modern studies a new pattern is emerging. For high social status individuals we see the exact same pattern as old studies, but as we go down the social ladder the picture change. For testosterone the current findings seem to link it to increases learned behavior when a individual has their social status challenge. That response in baboons is always more violence, but directed at the nearest individuals below them on the social ladder. For estrogen you see a drop in violence as you go down the social status ladder, while isolationing behavior goes up.
An other relative recent finding is that hormone secretion increase in men after winning social status. Unsurprising that correlated in animal studies when males has the highest probability of reproduction.
Hormones are useful when explaining behavior linked to reproduction, and in that you do get some gender differences. I would however be a bit cautious. A good talk I would recommend is https://www.youtube.com/watch?v=2bnSY4L3V8s&t=30m17s as it explore this a bit further.
And sexual interest. Hormones actually have many effects, but those are the most immediate/obvious ones.
You could view this as "no difference" though, as your gender identity would not rely on a male or female phenotype.
But like anything involving gendered characteristics in humans as expressed in the world, the range of observed behaviors within the two populations overlaps so much it’s hard to draw meaningful conclusions about any individual by differentiating the two.
Based on my personal experience and from talking to others who have medically transitioned, the difference between “inherently male” behaviors and “inherently female” behaviors can be nearly entirely explained by hormones. As in, you change hormones and the behavior changes: testosterone tends to cause elevated anger / aggression while estrogen tends to make people more neurotic with a wider range of emotional response. That’s it. The rest of the differences are either personality or environmental / social factors.
[1] https://www.nature.com/articles/s41598-017-17352-8