The lede I buried in this post is that we hired more-or-less resume-blind. I had final call on every hire and went through some effort not to look at resumes. On our first-calls, I'd ask some questions that would give me background hints about candidates, but that was primarily to tune my spiel (I didn't want to explain blind SQL injection to someone who'd spent 3 years as a pentester already, for instance).
If I had it to do over again at Matasano, I'd have done it formally resume-blind. Only Dina would have had access to resumes, and interviewers would be forbidden to ask resume questions. The process would have worked better if resumes were firewalled out completely.
There was no culture fit score; in fact, if you gave subjective feedback ("passion", "confidence", "work ethic") your feedback was likely to be discounted. We were proud of the culture diversity we managed to scrounge out of our candidate pool; to have team members with kids who needed to keep strict hours, and team members who'd roll in at 11 and still be there at 8 working on a pet problem. We had drinkers and non drinkers, college students and people with 15+ years of dev experience.
You could get booted from the process for being overtly an asshole (I, for instance, might not get hired), but that happened very rarely and with crystalline clarity.
Don't get me wrong, I understand and agree with your reasoning. I'm just curious how transparent it is to the other side.
So really "what did you study in college?" needs to come off the list of things you ask (at least as part of the filter).
on the reverse of that, i have sometimes avoided topics that the candidate was familiar with because i didnt see relevant experience on a resume. that was a major miss, and ive tried to avoid using resumes for shaping the interview topics since.. i see the resume now as more of a signal of what the person is interested in (since most of us carve huge swaths of our experiences out, and tailor them for the job we apply to anyways).
Two questions for you.
What is your experience hiring fresh college grads? Is the process different? Since one of goals of the process seems to not arbitrarily adjust it for every candidate, but rather stay consistent, do you think it makes sense to adjust it for someone who just graduated. They might have been learning computer science type stuff, data structures, did a few group projects and have certainly been practicing the typical interview puzzles, fizbuzz and O(n) times of the all data structures.
How do you account (or do you account at all) for culture fit. That is a fuzzy and slightly dangerous area. Is there a score in that regard? There are some people who are brilliant but they do not work well in a team, or rather would not well in one particular team. This is course full of danger as "culture" fit often becomes an opaque discriminator for racial, sexual and other stereotypes. "He's just not a good culture fit" is such an overload euphemisms that I feel kind of sleazy just saying.