It saved us enormous amounts of time at Matasano. When we designed our work-sample rubric, our #1 concern was filtering out people who interviewed well and/or had great resumes but weren't worth a billable hour on a real project, but our #2 concern was interviewing faster and getting results back to candidates quickly. (We didn't discover the best reason to do work-samples --- discovering hidden talent --- until after we'd started doing them).

With standardized interviews and challenges, our whole hiring pipeline became entirely mechanical. We could send our a challenge and its instructions in 2 minutes, and evaluate the response the next day (or week, or whatever) later in less than 5 minutes. 7 minutes, to produce a technical evaluation that crushed a competing battery of tech interviews by several senior staff members that took 4-5 hours to do.

Aha! I sort of like that, then. Assuming of course that it actually does work.

Back when things were simpler, we had a multiple guess questionnaire that did the similar. For onsites, we then looked for evidence of rigor and thoroughness, the ability to estimate how much communication was required, had they deployed real systems?

You understand that it sounds like you "fixed" hiring, and so we're all sorta skeptical, right? If you could bottle that, there's your billion.

> You understand that it sounds like you "fixed" hiring, and so we're all sorta skeptical, right? If you could bottle that, there's your billion.

Announcing Starfighter https://news.ycombinator.com/item?id=9173939

I am talking my book in the sense that my business works a lot better if companies get saner about how they hire people, but Starfighter is not the commercialization of the idea I'm talking about here.

Did you ever consider the candidate in all of this?

Yes: like I said, the goal was to get results back to candidates faster.

The biggest complaint we'd been getting from candidates before work samples was that our process took too long. Our goal became to get the entire process done under two weeks (from its worst point, a month or two earlier, of 1.5 months, when someone finally posted an anonymous complaint to Glassdoor about us).

After iterating a couple times, we could reliably go from first contact to hire/no hire, with a motivated candidates (ie: one who would work with us to schedule the interview process aggressively) inside of a single week.

Also, see: https://news.ycombinator.com/item?id=11328090

The process I'm advocating for is far better for candidates.

As a candidate, I would be a little wary of this process. You're asking me to put in multiple hours of work, but you only put in 7 minutes. So the incentives are misaligned: you are incented to give these challenges to many people, even if they have a low chance of passing through. But as a candidate, I don't want to sped multiple hours if I have a low chance of passing through. This is one reason why full-day on-site interviews aren't so bad - if I've gotten to that stage, I'm probably pretty likely to get an offer.

What do you do to ameliorate this concern?

First, I don't concede the idea that this is a concern I need to ameliorate, because the work-sample process consumes fewer hours of candidate time than the conventional interview does, and, better still, consumes those times as, when, and where the candidate chooses to make them available: an hour a night during the week, say. Do the work from your favorite quiet bar. Do it during your coffee break.

I am spending a lot of time these days talking to people interviewing in the valley, and what I'm seeing is that the norm candidates are subjected to is 7+ hours of 6-7 on-site interviews. Candidates have to go through all the interviews, even if the first interview has effectively ejected them from the process.

Compared to that horrid process, I don't believe I have to justify anything about my process.

But, if you read downthread, you'll see that we in fact did a lot to ameliorate the (bogus, I think) concern that we were incentivized to soak up hopeless effort from lots of people.

I was thinking something similar. When one party can waste the other party's time at little cost to their own, the situation can be abused. If a job ad wants you to submit to a test before you even talk to a hiring manager, this is a signal that the employer doesn't care about wasting your time. I can see the advantages to automating the hiring process but as a candidate I am less inclined to engage with a party that has no "skin in the game".

Think of the design world:

Potential client "A" asks multiple desperate artists to work on spec in hopes that they will get the commission.

Potential client "B" call you up and talks to you, sends you some napkin sketches and generally engages with you for an hour before asking you to do a design.

Both clients want you to do a design (test) but one clearly doesn't have any skin in the game. Assuming you're a decent designer, which client do you respond to?