[author here]

I actually agree with your thesis! You're right that candidates are only willing to put in this level of effort if they are really interested and consider your company a "top 5%" big opportunity.

So it's critical that you convince them that you are in that top 5%! You don't want an employee to join you because their top choices turned them down -- you want to be their first choice.

We regularly won candidates away from Google, Facebook, Uber, Twitter, etc. If you want to build a great team, you're going to need to be able to do this too.

Re: code reviews -- yes, we gave everyone a thorough code review.

>You don't want an employee to join you because their top choices turned them down -- you want to be their first choice.

Why?

Unless you're offering FAANG levels of compensation, isn't it pretty arrogant to think you're anyone's top choice?

Candidates can easily lie to you about this. In fact in my last job hunt I was instructed to lie about this by recruiters who are actually being paid by the company I'm lying to.

I told every company I interviewed at they were my top choice.

If it's "arrogant" to think that a startup could be anyone's top choice, then startups would never be able to attract any good talent.

It's not arrogant. It's simply creating a process where you can filter out for folks who are truly excited to build what you're building and believe in the potential future impact. It might also turn out to be a big payday, or not.

Not everyone is simply chasing that 400k TC every year. If you lie about your excitement, a panel of folks who have went through the same process will easily sniff it out.

Some people are better at pretending to be excited than others. Recruiters for example are very good at it. Not all engineers are bad actors.

Well, then perhaps the test is working as intended! Candidates who aren't sufficiently excited about your company will drop out at this stage.

This doesn't make any sense.

Your intention is to get candidates to fake excitement or drop out?

I think the idea is that it's easy to fake excitement until you're asked to spend six hours on a take home test. At which point you'd actually have to be pretty excited to do it.

Or desperate.

As someone who has given take-home tests let me answer these for you:

- Unit tests? Yes. It will increase your standing.

- Docs? Absolutely. A must have. You will not be considered really otherwise.

- Deployment? See above.

- Support 5k requests? Not unless that is what the test is testing for, pay attention during phone interview about the role and it's requirements, the test will look for this and other generalizations.

- Clean code? Very, very important. Don't try to show how smart you are, show how audience aware you are. Most likely you will be working on a team, a team doesn't want a cowboy programmer.

Something you didn't mention but bodes really well for anything with a front-end interface: hosting. Executives hate running things themselves, so a hosted version will put you at the top of the list.

It's probably worth mentioning that any take home worth its salt will explicitly list what criteria you're being assessed on.

It's crazy how many coding tests are poorly designed and waste a candidate's time. I'm putting together a list of good practice for test design here: https://candidatecode.com/articles/coding_challenge_best_pra...

Feedback and contributions welcome

A lot of the time they are left out to see how experienced the individual is working in the environment. What do they think is the most important (outside of the outlined criteria)? Treat it as if they are testing your experience, not your literal skill in pumping out code. Anyone can pump out code.

The issue there is the problem starts to become unbounded for the candidate; they can't tell what you're looking for, so they don't know how to prioritize their time.

That's a problem even if you have reasonable expectations yourself, because the candidate probably has experienced some tests where the expectations haven't been reasonable. For a (real) example: marking candidates down for not having an analytics integration.

How does your candidate know that's not important to you, even though it was important to the last company?

> testing your experience, not your literal skill in pumping out code

Past a certain point, it gets much more efficient to discuss experience in an interview setting. Good coding tests should be designed to test the skills you care about most to produce a strong hiring signal for the next phase - it's difficult to showcase a full decade's worth of experience in 4-8 hours' worth of coding challenge.

> Anyone can pump out code

I've found that to be depressingly untrue amongst a significant percentage of engineering applicants.

I like how this is what interviewers are looking for and then you get hired and there are no tests, the code is a mess and the performance sucks.

And they ask a hard algorithmic/CS problem then your job is to build CRUD interfaces

You nailed it.

That's why they look for it ;)

Clean code is so subjective it hurts. Being audience aware basically means trying to guess the company and interviewers coding style from minimal information.

Great story. Instead of looking as recruiting as a cost center, make it a marketing investment. Spend hours with someone, give them a good experience, make them an ambassador for your brand and an external sourcer for their friends and their future self. Use them as practice for your staff to get better as communicators, mentors, code reviewers.

Yes, exactly.

This is also a double-edged sword, if the experience is horrible, you can be sure none of their classmates will apply to you or even their school if you blew it exceptionally.

There is an article by Joel Spolsky about Hiring great engineers that touches on this point also: https://www.joelonsoftware.com/2006/09/06/finding-great-deve...

> Even though only about one in three applicants who make it to the in-person interview stage passes all our interviews, it’s really important that the ones that do pass have a positive experience. Even the ones that don’t make it go back to campus thinking we’re a classy employer and tell all their friends how much fun they had staying in a luxury hotel in the Big Apple, which makes their friends apply for an internship the next summer, if only for the chance at the trip.

> I have discontinued multiple interviews because they demanded 4-5 evenings of work for free on short notice, if I am going to spend 10-20 hours per company I have to take two weeks off from my current work to do it.

Honest question: Would you feel differently if these companies had 1-2 days of onsite technical interviewing instead of take-home problems?

In practice, I rarely get pushback from candidates for our 4-hour take-home problem. We provide clear instructions about expectations, constraints, and boundaries. All candidates have thanked us for allowing them to work on their own time in their own coding environments.

I would, because interviews require someone from the other organization to be there. Time is a valuable resource, and as long as the employer is showing their commitment, I have no problem reciprocating. Skin in the game is the key.

Why does "skin in the game" matter? Why is that a useful signal about whether this company will be a good employer or not?

It's not a useful signal about if they are a good employer, but it is a good signal for how likely they are to hire you: asking everyone who applies to take several hours out of their life to do your test is free. Having someone from your team interview them for several hours very much is not so you'll be more careful selecting candidates you might actually hire instead of just taking a shotgun approach.

For me it's a useful signal, it means they value my time equally as theirs. I'm happy to do the take-home test after the onsite one when they only testing that I really know what I said I know but I'm at the finish line. A lot of companies require it right after the phone screening(usually with an HR person who knows nothing). I burned myself that way before. After finishing the take home test on the on site interview after 5 minutes it was clear they were looking for an ops person advertised as DevOps.

Right, having the employer invest something means they'll be more likely to check for a good fit before reaching for homework as a filter.