Thing is though, the size of people's gametes has no bearing on their weightlifting ability. Rather, secondary sexual characteristics do. Those characteristics are very strongly associated with gamete size, but not perfectly. So classifying sex purely on the size of people's gametes will lead to a few odd outcomes.