From the article:
> For gender-neutral profiles, we included GitHub users that used an identicon, that Michael’s tool could not infer a gender for, and that a mixed-culture panel of judges could not guess the gender for.
Of course, there is still the issue that they have effectively limited their sample to people with Google+ accounts which may affect the results of the study. Given that men's acceptance rate also dropped when their gender was identifiable (but not by as much) gives credence to the idea that there might be a flaw in their Github profile analyzer.
Further down they then distinguish between contributors where the gender can be inferred from looking at their name & profile picture. Splitting the group of those 35% which were identified via Google+ into two separate groups - identifiable vs. non-identifiable.
Which leaves me frustrated. It seems I want things that mostly do not exist.
I find this suspect, because, anecdotally, I can look almost any author of a PR and determine their gender to a high degree of certainty. Probably well over 90% of the time, just from their name + profile picture + handle. Try it for yourself..look at the latest commits on a random project and see how obvious the genders are most of the time.
So the claim that "when a woman is identifiable, PRs are merged less" is totally suspect, because they themselves can only identify the genders of a small percentage relative to what a normal human can identify. If people can identify the genders way more often and accurately, then the claim being made is bogus. Perhaps there is a correlation of strongly signalling your gender (to the point where an inaccurate method of gender-identification has no problem) to being a below average developer.