Plus, if you wanted to have more accurate data, you could ask more accurate questions, e.g. "are you able to give birth" or anything like that. Now, people might refuse to answer, but that's their right.
Being able to conceive is not a simple yes or no answer. If you simplify beyond the conversation with your doctor, you need to make unfortunate assumptions. That’s why it’s so thorny.
Some people believe that trans rights mean that suddenly everyone will try to claim they're "actually" men or women without any real skin in the game. I find this line of reasoning, no matter what your opinions about biological vs. other types of gender or about pronouns are (personally, I find pronouns besides he, she or they to be slightly ridiculous), to be akin to "gay rights mean people are going to stop having children" types of arguments. It's just not such a big deal that it's worth starting a culture war over IMHO.