I've long wanted to see in general some more experimental testing of selection variations. What if YC (or some other funder's) candidates were just selected completely randomly from the applications? What if they were selected solely according to some dumb criterion, like take everyone with the most degrees, or the longest CV, or the most GitHub LoC? What if they were selected purely based on the applications (without the dumb-criterion requirement) but without interviews? For a few tens of thousands of $$, someone willing to try those kinds of things out could get some pretty interesting information on how reliable different selection methods are.
My own hypothesis is a negative one: that beyond screening out a few obviously-bad candidates and taking a few obviously-good candidates, the bulk of the YC selection process is randomly related to outcomes, and the YC mentoring/contacts/press/etc., rather than predictive value of the selection process, is the main driver of their generally strong outcomes. But I can't prove that. :)
Also, you have to consider how much quantitative and qualitative experience YC has accumulated, the partners are pretty good at telling in a couple minutes conversation if you're a strong founder. This advantage would be lost with blind interviews.
This is the part I doubt, though, if by "strong founder" you mean "statistically more likely to exit successfully than people selected according to much simpler 'dumb' criteria". These kinds of claims to predictive ability based on un-quantified holistic properties like "experience" rarely hold up under scientific scrutiny.
http://en.wikipedia.org/wiki/Pygmalion_effect#Rosenthal.E2.8...
2.) Paul has admitted to being susceptible to the Mark Zuckeberg effect, at least he was honest about that and should be respected for the fact that he realizes that. Most VC's i believe also fall into this trap but don't admit it.
3.) Now what are we going to do about this? Shrug our shoulders and just say this or that won't work or get to trying solutions and iterating on that?
How many are asking for this? Where are your stats?
I would disagree strongly in that YC has a measurable financial risk of excluding potentially profitable founders solely for meaningless cultural woo woo reasons. For example if some Finnish dudes conduct perfectly normal business transactions nude in the sauna, a prudish American who refuses to participate has an obvious measurable economic loss solely because of irrational cultural woo woo. Now extend that far out example into female communication style.
Now what would work, or at least would be interesting, is having a female partner interview female founders separately from the male partners then study the female partner's impression vs male partner impression. I don't suspect there would be a huge difference; but at least this would be a somewhat more effective way to test the proposed effect. For my ridiculous made up example, you'd need a Finnish partner; probably easier to run this test on the somewhat easier to acquire and categorize male vs female test subjects.
While I agree that blind applications would be somewhat tough for startup founders, conducting seems like a bad example. You could fairly easily judge the resulting music without being able to see the conductor.
Even with world-class orchestras, where performances are regularly put on with guest conductors after only a few hours of rehearsal time, no permanent conductor would ever be hired on the basis of merely listening to their music. It's a leadership position. (Unlike orchestra players, where it really is more directly about musical proficiency.)
When you evaluate a team, you need to be able to judge their confidence, see how they interact with each other, get a feel for the trustworthiness, the way they look at you when they answer a question, and so on. If you can't see them, and their voice is distorted, then you might as well just ask for a slide deck and forgo an in-person interview altogether. Which doesn't seem like a good idea.