Maybe it's okay for a man to be the spokesperson for women's issues, or a white person to be the spokesperson for black issues, but it's important to realize you're speaking for someone else. It's really hard to do that better than that someone. Maybe I'm a better public speaker. Maybe people are more likely to listen to me. Maybe I even have a better understanding of power dynamics and sociology and tech culture. But it's also really, really critical to have an understanding of the perspectives of those I'm speaking for. That's hard. And in this situation, seems like a bunch of those people seemed to feel that wasn't the case.
Also, anecdotal experience, even by large numbers of the population, should not be taken as anything significant without a scientific endeavor to interpret them. Consider the numerous parents who will state "As a parent..." when disagreeing with an expert on child development. Even if thousands of parents disagree, even if the expert has never been a parent, an immunologist saying children should receive vaccines is still correct.
In all other fields, the expert lacking personal experience has the better perspective than part of the population impacted with their own personal anecdotes.
I think this would be more accurate with the "a man" part. The spokesperson for women in tech shouldn't be. Women in tech aren't a monolithic group such that a spokesperson is sufficient.
Why? You explanation shows why women are perfectly valid candidates; they don't need a man. But why would a man be a bad choice? Why do they need a woman?
If you're a man (like I am!), and you want to help women in tech (like I do!), and someone approaches you to speak at a conference about that, the thing to do is to recommend some awesome women you know to speak instead. Instead of talking about how women need support in this industry, support an actual woman in the industry.
Otherwise you run the risk of coming off like Wadhwa did. He looked like a carpetbagger (http://en.wikipedia.org/wiki/Carpetbagger) -- someone who had identified a hot trend and jumped on it to raise his own personal profile. And if you're a man who sincerely wants to help women, the way you do that is by helping them raise their own profiles, rather than using their problems to goose your own.