News for hackers. Stuff that matters.
Re:The Utter Uselessness of Job Interviews by tptacek on Sunday April 09, 2017 @02:06PM (#14073527)
This is a good and important question. Also: while I have a lot of respect for people at Google trying to innovate on hiring, make no mistake: Google's heart is in the right place, but they aren't at the forefront of structured hiring, and their hiring processes are notoriously capricious.

The reality is that generating good questions for a structured interview is difficult. You can't just pose a programming problem. As you've noted, most programming problems have multiple good answers. Differentiating between multiple good answers from different candidates re-introduces subjectivity. Your brain would rather convince you that one valid answer is less good than another, even when it's not, that to admit to you that it can't differentiate or generate a narrative for you. The part of your brain that generates narratives is incredibly powerful and does not care about how accurate your hiring process ends up being.

What we tried to do was create questions that generated lists of facts. "Spot all the errors in this code" would be an example of this approach (but none of the three we used). We went into the process wanting to embrace epistemological uncertainty, generating a historical trail of data that we could retrofit to candidate performance.

In the end, work sample testing was so much more powerful a predictor for ourselves that we never fully got around to analyzing the data. Sometimes we'd get candidates that clearly generated inferior "lists of facts"; I think there may have been 1-2 instances where that outcome actually overruled work-sample testing delivered prior (out of a few tens of engineering hires and probably ~100 interviews).

Re:The Utter Uselessness of Job Interviews by peripitea on Sunday April 09, 2017 @04:32PM (#14074224)
Making sure I understand you: When you refer to work sample tests here, that refers to the crypto challenges and things like that that you published at Matasano? And you're saying that was much more predictive than the list of facts methods, right?
Re:The Utter Uselessness of Job Interviews by tptacek on Sunday April 09, 2017 @06:25PM (#14074663)
That's what people ordinarily assume I mean, but while our work sample challenges were similar to the cryptopals and Microcorruption stuff, they were not the same, or even derived from them. They were designed specifically to qualify candidates, and in fact predated our public challenges.
Re:The Utter Uselessness of Job Interviews by sgustard on Sunday April 09, 2017 @08:15PM (#14075065)
Just to further clarify, by work sample you do NOT mean an example of previously produced work? This still seems fickle: my dozen years of work output (for example, building and running a site with perfect uptime for millions of users) is not as valid an indicator of my future performance as how I happen to score on some arbitrary timed test?
Re:The Utter Uselessness of Job Interviews by tptacek on Sunday April 09, 2017 @11:45PM (#14075837)
Absolutely not. Samples of previous work are deceptive at the best of times.
Re:The Utter Uselessness of Job Interviews by peripitea on Sunday April 09, 2017 @08:01PM (#14075018)
Ahh, interesting. Would you mind sharing an example of how they were different, or how the fact that they were specifically for qualification changed the thought process? I'm working on making tech interviewing better and am fascinated by this stuff.
Re:The Utter Uselessness of Job Interviews by leksak on Sunday April 09, 2017 @08:45PM (#14075182)
NDAs? Everything I've done yet is proprietary and judiciously covered by various binding legal agreements.

I wouldn't breach such a contract nor should you hire someone that would be willing to do so. The sole exception being in the cases where it would be the best for the public interest, i.e. whistle-blowing.

Re:The Utter Uselessness of Job Interviews by mbesto on Sunday April 09, 2017 @03:37PM (#14073980)
> but they aren't at the forefront of structured hiring,

Is anyone though?