http://www.paulgraham.com/say.html
A Harvard President lost his job over remarks similar to what you just made. And yet, we could observe that in aggregate, men and women have different hormone balances, biochemistry, reproductive systems, feel different degrees of different emotions, account for very different sorts and frequencies of crime, purchase very different products and services, and so on and so forth.
So yes, while you are factually correct in many ways, I am surprised you are still standing. The idea is not a pleasant one to most people. Perhaps because it can too easily be carried too far, or misapplied, or that the generalization will be mis-applied to a particular individual who will be offended, and certainly our society puts a very high premium on not offending anyone. Or maybe I am off in my thinking, but truth aside, I remain surprised that you have not piled on for making such a comment.
There are very reliable statistics that show pay discrepancies in the favour of men. For discussion of that to be taboo would be very bad.