News for hackers. Stuff that matters.
Re:Girls Who Code by gfodor on Sunday December 29, 2013 @02:17PM (#6980815)
"Culture fit" is a vague enough term to not really be able to say much about it. Are there companies who use it as a way to hide behind discrimination? Sure, almost certainly. But in my past experience the amount of times we rejected a candidate due to "no culture fit" was quite low and was for reasons that were of the flavor: "this person sounds like they totally caused drama in their old workplace" or "this person admitted to bending the rules in a way we find ethically dubious" etc. In other words, there are certainly alternative criteria for "culture" fit that are legitimate things to consider and are not discriminatory.
Re:Girls Who Code by tptacek on Sunday December 29, 2013 @02:39PM (#6980914)
This speaks to how deranged mainstream software hiring practices are.

To wit: take a bunch of practitioners more or less at random and arrange to have them serially interview candidates. Apply virtually no structure to the interviews themselves; most of those practitioners will be free to generate their own questions, and if any guidance is given at all, it will usually take the form of "assess the candidate's abilities along axes X, Y, and Z" (for "X" of the form "Rails" or "Django" or whatever, &c). To that haphazard scheme, strongly imply that those practitioners are not merely expert at assessing the abilities of others (something that does not follow from the mere observation that the practitioners themselves are competent in the field), but also psychologists capable of extracting from a 45 minute adversarial interview the "culture" and mentality of candidates who are so nervous and upset from being judged by hostile strangers in this crapshoot process.

Software hiring is widely regarded by hiring managers as a nearly random process; at best, at savvy companies, the house might have an edge over time. And the cost of that edge is a system that is inhumane to virtually everybody, with the possible exception of a small elite of sought-after developers.

Re:Girls Who Code by andrewljohnson on Sunday December 29, 2013 @02:53PM (#6980994)
How does Matasno's hiring process work? I don't think I've heard you comment on hiring except to say everyone does it wrong.

In my company, we've hired everyone based on remote/part-time work, and paid tryouts. This seems to work well, but I don't feel like I can hire people who currently have full time jobs this way. I'm not sure how best to judge candidates who I can't observe working, besides a standard battery of technical interviewing and a lunch with the team.

Re:Girls Who Code by tptacek on Sunday December 29, 2013 @03:07PM (#6981062)
I'll try to condense it as much as possible, and add the caveat that our process is imperfect:

1. Build outreach programs that are open-enrollment and broadly targeted (for instance, the crypto challenges aren't targeted at security industry events).

2. Explicitly overcommunicate the whole hiring process to candidates up front and sell the job, before attempting to screen. Actively try to correct for the adversarial nature of the process.

3. Drastically reduce the impact of phone screen interviews.

4. Administer standardized work-sample tests to all candidates regardless of previous experience; those tests have quantifiable outcomes (which we record).

5. Administer standardized on-site interviews with quantifiable outcomes (right now, they're "exercises", but their form might change over the long term).

The only point in our process where interviewers have a lot of flexibility in terms of what they do is the phone screens. We use the phone screens to generate predictions for how candidates will do on the rest of the interview, but we don't screen with them. (We would if the phone screen revealed a candidate to be a crazy person or a jerk, but I don't think that has ever happened).

So far, the things that have been most noticeably successful for us:

* Standardized, quantifiable results that allow us to generate a model of what a successful candidate actually does, rather than a gut feeling based on how well they answer questions, or how "confident" they are, or how "well spoken", or how impressive their resume.

* Outreach to the industry as a whole, not just to the small sectors of the industry we believed a-priori would generate the best candidates. Some of our best people have come essentially "out of nowhere", and recognizing that fact I'm especially interested in optimizing for that.

It is thrilling --- best word I have for it --- to find someone who's had no relevant job experience, hire them because they do unexpectedly well on a series of quantifiable metrics, and then see 6-9 months later that they are, say, showing us how to use BKZ lattice reduction and fourier transforms to exploit elliptic curve nonce biases.

Re:Girls Who Code by yummyfajitas on Sunday December 29, 2013 @03:57PM (#6981353)
This sounds like a vast improvement over most normal hiring processes. A question though - how do you avoid legal liability from (4) and (5) in the event that they cause a disparate impact (see Griggs vs Duke Power, etc)?
Re:Girls Who Code by tptacek on Sunday December 29, 2013 @04:05PM (#6981389)
I don't think about it at all, and for good reason: nothing that our process does could possibly be more risky than the crapshoot process other companies use. To wit: I know for a fact that my process isn't going to be dumb enough to ask a woman whether she's going to be able to manage her child care responsibilities and still be able to answer emergency work calls.
Re:Girls Who Code by yummyfajitas on Monday December 30, 2013 @03:22AM (#6983674)
The danger comes from the fact that courts generally give deference to ad-hoc processes, whereas systematic processes are inherently suspect. That's why interviews are fine but IQ tests are legally suspect.

As another example in this general line of thought, compare Grutter v. Bollinger and Gratz v Bollinger. Point-based racial discrimination is illegal, but "holistic" "individualized" processes where race is just a "plus factor" are legal. (Only discrimination in favor of non-Asian minorities, of course.)

I agree that this legal doctrine is insane.

Completely tangentially, since you mentioned relevant questions which are illegal to ask, I'd love to see an economic analysis of this topic. Specifically, the classical Akerlof lemon market paper assumes a complete inability to measure quality. But in the hiring market there are only a few dimensions for which one is unable to measure quality. That complicates the analysis significantly, so I'd love to gain some intuition on the topic.

Re:Girls Who Code by temphn on Sunday December 29, 2013 @05:59PM (#6981892)
How many women and non-Asian minorities does Matasano end up hiring with that process?
Re:Girls Who Code by defen on Sunday December 29, 2013 @04:30PM (#6981478)
The point of Griggs vs Duke Power is that a work sample test would probably need to be "reasonably related" to the job at issue, if it has a disparate impact on minorities (meaning minorities pass at a lower rate). I have no idea how one proves in court that a test is "reasonably related", because I'm not a lawyer :-)
Re:Girls Who Code by tptacek on Sunday December 29, 2013 @04:39PM (#6981517)
Yeah, I'm familiar with it; my point is that the ad-hoc procedures companies use in place of work-sample testing are much more perilous. I've been a witness to more than one legal action that resulted from them.
Re:Girls Who Code by defen on Sunday December 29, 2013 @04:47PM (#6981562)
I see. For some reason I read your statement as being about disparate impact rather than ad-hoc hiring procedures.
Re:Girls Who Code by WalterBright on Sunday December 29, 2013 @02:58PM (#6981016)
All decision making processes with a human in the loop are going to be random, arbitrary, capricious, etc. But inhumane? about a job? It actually seems very humane. After all, only a human can look past the rules and see that a candidate is someone special that would be great for the company despite not matching the criteria.

Would you rather be selected for a job by a computer that follows inflexible rules embedded in software? That would be inhumane.

Re:Girls Who Code by tptacek on Sunday December 29, 2013 @03:09PM (#6981080)
I am telling you that after watching the Nth person go through an on-site interview process that we had already actively tried to program as positive and encouraging, and seeing them physically shaking in the conference room, and then thinking back on the last time I had needed to endure a job interview and how I felt, that yes, the process is inhumane. It demands of candidates that they endure a deeply uncomfortable process of being judged in 45 minute snapshots by strangers who often by all appearances seem to have rejection as an overt goal.

Your last question presents a false dichotomy and so I won't answer it.

I am, incidentally, a fan of your comments on HN. I'm just not interested in an argument where we're both going to be talking past each other.

Re:Girls Who Code by ericd on Sunday December 29, 2013 @04:42PM (#6981534)
Yeah, I completely agree - I absolutely hate interviews, to the point where I would prefer to do a large amount of work on alternative methods over a full day of interviews.
Re:Girls Who Code by gfodor on Sunday December 29, 2013 @02:43PM (#6980928)
Is it the case that you do not screen candidates based upon any other criteria than skills-based assessment in other fields? That assumes people are robots. I don't care what field you are talking about, people are always going to screen for "culture fit", since it can be defined to mean anything you want. It's "can I work with this person", "does this person seem trustworthy and honest", etc. The only difference is that it some places acknowledge this aspect of hiring, give it a name, and try to objectively assess it openly.
Re:Girls Who Code by tptacek on Sunday December 29, 2013 @02:48PM (#6980959)
You can complete the sentence "people are always going to ________", especially as regards software hiring, with all sorts of crazy shit. Hiring is broadly and deeply fucked up. Very few organizations do it especially well. The median companies does it abysmally. But we don't notice, because we develop other systems to compensate for how bad we are at getting the right people in the door.

That's my point, why I bring up how bad hiring is. "Widespread best practice" in hiring does not imply effectiveness. Most companies are terrible at hiring. They just get away with it because other aspects of those companies are good enough to compensate.

Re:Girls Who Code by philwelch on Sunday December 29, 2013 @03:00PM (#6981022)
"Culture fit" is a terrible word for it. What you do is you explicitly lay out the principles you expect people to follow and conduct behavioral interviews to find times they've followed those principles in the past.

Here's an example: let's say you're a typical startup, and you need to move fast. You need to think fast and you need to act fast. Let's say you've made the decision to hire people around that principle. You can ask someone about how they've coped with short deadlines, or to tell you about a time they had to get a lot done in a short period of time, and you judge them based on their responses to your questions.

Generally there will be a number of different principles, and there are also ways you can overdo them, but you want to decide on the right balance and the right mix and then conduct behavioral interviews to measure candidates against that.

The wrong way to do it is to make subjective judgments like, "I want to work with this person" or "I can really have a conversation with this person".

Re:Girls Who Code by bermanoid on Monday December 30, 2013 @01:23AM (#6983345)
If someone rubs you wrong personally, why in the world would that not be relevant, especially on a small team? Sure, you don't have to be besties or anything, but being able to at least converse with someone makes any work relationship go a lot smoother, and inability to do so can cause serious problems.

I'd never let a good culture fit override a poor technical interview, but I've seen the consequences of hiring a brilliant programmer that happens to be an asshole, and it's not pretty. Am I misinterpreting what you're saying, or do you really think social skills should be entirely ignored when it comes to hiring?

Re:Girls Who Code by philwelch on Monday December 30, 2013 @01:48AM (#6983420)
> If someone rubs you wrong personally, why in the world would that not be relevant, especially on a small team?

Because business decisions should be made based upon facts and data, not based upon hunches and gut feelings. Going with your gut basically means throwing out the data and letting all of your personal biases decide the matter, consciously or unconsciously.

If it's a question of being able to work with others, then you can develop principles and behavioral interview questions around what you want your coworkers to be like. That's a bit of a pain in the ass, but empiricism was always more of a chore than just following your gut instinct.

Re:Girls Who Code by bermanoid on Monday December 30, 2013 @02:45AM (#6983591)
Ok, I can buy that - your point is that you should be attempting to get an unbiased estimate of how well someone works on a team, not that it shouldn't matter. Totally reasonable. Much more difficult, but a worthwhile goal.

I feel like it might be almost impossible to remove gut feeling from the equation alogether, though, since a lot of the problems that wreck teams really do come down to "X really doesn't like Y's personality". At some level, isn't a negative gut feeling expressed by one of your team members as valid a data point as you can get to predict how well the person will work out on that team?

Re:Girls Who Code by philwelch on Monday December 30, 2013 @12:13PM (#6985510)
People who experience personality conflicts with each other can be civil to each other and interact in a professional capacity. Not being able to do that might be a problem, but we're all adults here.