Well, we don't but attendance rate are already taken into account. There might be other cultural problems though like being in a minority being difficult or more likely girls just doing worse in competitive social environments which are typical for sports.
One can even argue that girls already have advantage in selection: assuming equal talent distribution in boys and girls populations and assuming most talented children from both make it to the competitive chess, the girls should do better (because less of them are selected and therefore more talented ones on average).
If anything it's likely that once selected (that is starting to play competitively) the environment is less welcoming to girls. On the other hand girls get more opportunities than boys (dues to women only tournaments, titles, world championship).
While I think the hypothesis that environment is less welcoming and thus inhibit development of women players I have trouble seeing how this can be tested.
Aptitude testing shows[1] that while females tend to cluster more towards the average, male populations produce more outliers. There are both more male geniuses and male imbeciles than their female counterparts.
[1] for example http://www.aei.org/publication/chart-of-the-day-scottish-iq-...
This is the product of the different evolutionary strategy for males and females, the males being high risk/high reward, the females being conservative/average. I don't have a citation for that.
Humans are cultural creatures, cultural thinkers and movers, and in everything from chess to MMA we benefit greatly from growing up in a culture that is tooled to support our specific strengths. Not everyone gets that chance, and not all groups equally.