Discriminating based on fitness instead of gender is exactly the sort of thing one should do, if fitness is really what you are worried about.

That way, the occasional woman who is if fit enough can still do the thing that needs doing, and flipping genders on your birth cert doesn't game the system.