To wit: take a bunch of practitioners more or less at random and arrange to have them serially interview candidates. Apply virtually no structure to the interviews themselves; most of those practitioners will be free to generate their own questions, and if any guidance is given at all, it will usually take the form of "assess the candidate's abilities along axes X, Y, and Z" (for "X" of the form "Rails" or "Django" or whatever, &c). To that haphazard scheme, strongly imply that those practitioners are not merely expert at assessing the abilities of others (something that does not follow from the mere observation that the practitioners themselves are competent in the field), but also psychologists capable of extracting from a 45 minute adversarial interview the "culture" and mentality of candidates who are so nervous and upset from being judged by hostile strangers in this crapshoot process.
Software hiring is widely regarded by hiring managers as a nearly random process; at best, at savvy companies, the house might have an edge over time. And the cost of that edge is a system that is inhumane to virtually everybody, with the possible exception of a small elite of sought-after developers.
In my company, we've hired everyone based on remote/part-time work, and paid tryouts. This seems to work well, but I don't feel like I can hire people who currently have full time jobs this way. I'm not sure how best to judge candidates who I can't observe working, besides a standard battery of technical interviewing and a lunch with the team.
1. Build outreach programs that are open-enrollment and broadly targeted (for instance, the crypto challenges aren't targeted at security industry events).
2. Explicitly overcommunicate the whole hiring process to candidates up front and sell the job, before attempting to screen. Actively try to correct for the adversarial nature of the process.
3. Drastically reduce the impact of phone screen interviews.
4. Administer standardized work-sample tests to all candidates regardless of previous experience; those tests have quantifiable outcomes (which we record).
5. Administer standardized on-site interviews with quantifiable outcomes (right now, they're "exercises", but their form might change over the long term).
The only point in our process where interviewers have a lot of flexibility in terms of what they do is the phone screens. We use the phone screens to generate predictions for how candidates will do on the rest of the interview, but we don't screen with them. (We would if the phone screen revealed a candidate to be a crazy person or a jerk, but I don't think that has ever happened).
So far, the things that have been most noticeably successful for us:
* Standardized, quantifiable results that allow us to generate a model of what a successful candidate actually does, rather than a gut feeling based on how well they answer questions, or how "confident" they are, or how "well spoken", or how impressive their resume.
* Outreach to the industry as a whole, not just to the small sectors of the industry we believed a-priori would generate the best candidates. Some of our best people have come essentially "out of nowhere", and recognizing that fact I'm especially interested in optimizing for that.
It is thrilling --- best word I have for it --- to find someone who's had no relevant job experience, hire them because they do unexpectedly well on a series of quantifiable metrics, and then see 6-9 months later that they are, say, showing us how to use BKZ lattice reduction and fourier transforms to exploit elliptic curve nonce biases.
As another example in this general line of thought, compare Grutter v. Bollinger and Gratz v Bollinger. Point-based racial discrimination is illegal, but "holistic" "individualized" processes where race is just a "plus factor" are legal. (Only discrimination in favor of non-Asian minorities, of course.)
I agree that this legal doctrine is insane.
Completely tangentially, since you mentioned relevant questions which are illegal to ask, I'd love to see an economic analysis of this topic. Specifically, the classical Akerlof lemon market paper assumes a complete inability to measure quality. But in the hiring market there are only a few dimensions for which one is unable to measure quality. That complicates the analysis significantly, so I'd love to gain some intuition on the topic.
Would you rather be selected for a job by a computer that follows inflexible rules embedded in software? That would be inhumane.
Your last question presents a false dichotomy and so I won't answer it.
I am, incidentally, a fan of your comments on HN. I'm just not interested in an argument where we're both going to be talking past each other.
That's my point, why I bring up how bad hiring is. "Widespread best practice" in hiring does not imply effectiveness. Most companies are terrible at hiring. They just get away with it because other aspects of those companies are good enough to compensate.
Here's an example: let's say you're a typical startup, and you need to move fast. You need to think fast and you need to act fast. Let's say you've made the decision to hire people around that principle. You can ask someone about how they've coped with short deadlines, or to tell you about a time they had to get a lot done in a short period of time, and you judge them based on their responses to your questions.
Generally there will be a number of different principles, and there are also ways you can overdo them, but you want to decide on the right balance and the right mix and then conduct behavioral interviews to measure candidates against that.
The wrong way to do it is to make subjective judgments like, "I want to work with this person" or "I can really have a conversation with this person".
I'd never let a good culture fit override a poor technical interview, but I've seen the consequences of hiring a brilliant programmer that happens to be an asshole, and it's not pretty. Am I misinterpreting what you're saying, or do you really think social skills should be entirely ignored when it comes to hiring?
Because business decisions should be made based upon facts and data, not based upon hunches and gut feelings. Going with your gut basically means throwing out the data and letting all of your personal biases decide the matter, consciously or unconsciously.
If it's a question of being able to work with others, then you can develop principles and behavioral interview questions around what you want your coworkers to be like. That's a bit of a pain in the ass, but empiricism was always more of a chore than just following your gut instinct.
I feel like it might be almost impossible to remove gut feeling from the equation alogether, though, since a lot of the problems that wreck teams really do come down to "X really doesn't like Y's personality". At some level, isn't a negative gut feeling expressed by one of your team members as valid a data point as you can get to predict how well the person will work out on that team?