First of all, for the sake of clarity, most scientists differentiate between gender (a social role) and sex (a physiological characteristic)[1][2]. By "biological gender," I assume you mean, "The gender most often associated with the sex of the person."

Second, you should probably also know that sex is also not binary even on a cellular level and is much more complicated than most laypeople understand[3].

Those things said: differences between biological sexes are not at all controversial except for maybe a few fringe-y people on Twitter or reddit. You really shouldn't assume that the loud, anonymous people on the internet represent any larger group.

The actual argument from social liberals (in the US, where I live) is that there are biological differences on average, but that does not allow you to draw conclusions about a specific person based on their sexual organs.

There is also a long history of extremely flawed gender research in the social sciences, partly because it's very easy to design and conduct a biased social experiment that no one will ever reproduce.

1. https://www.who.int/gender-equity-rights/understanding/gende...

2. https://www.physiology.org/doi/full/10.1152/japplphysiol.003...

3. https://sciencebasedmedicine.org/sex-gender-and-sexuality-it...