He's suggesting that for a lot of people getting involved in open source it is a means to an end. Whether that is getting a job in a new field, increasing exposure or experience, etc...
Usually that end is something monetary or status related.
What he is then saying is that some parts of the population are more privileged than others when it comes to the free time and resources needed to achieve that. It's not necessarily just that they have the resources now, but that they may have had them in the past too (upbringing, access to education, encouragement, etc).
So the discussion of privilege shouldn't necessarily be seen as a negative force trying to tear down white men, but should inform our decision making when we take other people's hardships into account. As a white male myself I understand that I have had better opportunities than a large part of the population. I'd rather take that into account and try to improve the fairness in society, than to look away from it. What to actually do about it is another matter... But acknowledging that it's a problem is the first step.
This article provides more context:
http://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-an...
--
On privilege:
I completely disagree with the necessity of being engaged in a community to be a compiler developer or to learn any CS domain.
But stating that free time is a privilege is a bit absurd. You could be a 23-hour a day gardener and still write a compiler. It's not as if there's a time limit.
Dividing by gender is ridiculous---the first computer programmer was a woman.
All that it takes to become a compiler enthusiast is the understanding of languages and computers. Having a pencil and paper is a plus.
Alan Turing was gay but it didn't prevent him from inventing a form of universal computer.
People should be celebrated for their differences, not admonished for not having enough outlying characteristics.
Remember, the first programmers had neither computers nor compilers, and the first compiler writers had punch cards, so who exactly is privileged?