Unless you are the top company in your bracket, your completion rate will also not be so great for a take-home (people have limited time: they'll sort by how good your company is). Sure they'll do Google's take-home but local business that does local comp/prevailing wages? It's at the bottom of the pile. By the time they get to it, the best candidates will already have an offer from a better company.

That's why I prefer to bring people on-site as soon as possible, and do a little whiteboard (the point being that the candidate must... know how to code!). It's an artificial challenge that's really a pretext to have a technical discussion.

Only time I advise to use take-homes is when you are dealing with a massive number of applicants from institutions that have a poor signal to noise ratio. Bootcamps come to mind. But then you end up with candidates that have the homework done by someone else and it ends up not being a good filter either.

Apparently we should just review github OSS contributions and how many unit tests they write.

Sketch out rough program structure or system design or whatever

Vs

Write valid Python to solve some leetcode question

No, but speaking from experience - also don't reach out to senior, employed devs for a role, then waste their time with whiteboard leetcode quizzes to "make sure they aren't fakers or coasting, we have sooo many applicants".

In those cases, a conversation is really more valuable, we aren't interns.

The "context" (hints) they give different candidates can vary massively, perhaps by the end you've realised that your question is ambiguous, or you like the personality of this other candidate.

I've worked with folks who will definitely give attractive women more guidance throughout so they arrive at the "correct" solution and not see what they are doing.

I'm not sure. Ability to remain calm and code when shit's on fire can be fairly useful skill at some places (the smaller the company, the more valuable it can be). I've did that, coding emergency patches - e.g. to cache some hot paths when we've got unusual traffic that revealed bottlenecks that we've never hit (and noticed) before. Though I'd say it's a fairly different mental experience than coding at an interview.

Also, actually writing code on a whiteboard is a weird skill outside of academic roles. But most places I've seen asked to sketch some crude pseudocode or - better - draw wavy lines and boxes that represent processes and entities, then talk about it (and I think this is fine, whiteboard here is just a convenient tool).

And, of course, requiring to remember some algorithms from memory is a rarely useful skill. I've coded even in quite spartan conditions (right in a server room, with an ancient CRT monitor and keyboard, plain vim running on a 80x25 text console) but even then I had one or two Internet-connected devices that I could've used to look things up. If a candidate is allowed (or, better, even encouraged) to search for anything they want - it's all good, of course.

Outside of education, no one bothers to remember actual algorithms unless they've used them recently - everyone remembers names and key properties, and optionally remember some core principles (and then either just use them because they're already in a library or popular package, or go look for the implementation details).

This is the key. Not all pressure is the same.

Shit on fire at work is not actually personal pressure, it's pressure on the company. And to be clear, shit on fire at work tends to be a metaphorical exaggeration, where an emergency code patch isn't going to save the company from imminent bankruptcy. You're still going to have a job the next day, still going to have a roof over your head and food on the table. And your coworkers already know you and respect your work (if you're competent).

This is vastly different from being unemployed, suffering from profound financial worries about your future, and having complete strangers looking over your shoulder, prematurely judging you in the span of a few minutes, where these few minutes can determine your personal fate.

I've done quite a few emergency fixes over my career, but exactly zero of them were lose-the-job emergencies. Whereas every job interview is a lose-the-job emergency.

I keep myself in the interviewing loop at my company because I actually enjoyed the process when I joined. It was conversations over “executable” solutions. Now, we kick out so many candidates that can’t get a coded, running solution in the 45 minute slot.

I have tried so many times to explain the WILD asymmetry going on. That for us, we just have to interview the next candidate. To them, they have the enormous pressure of “if I can’t do this exercise then I have no income, no health insurance, no security.”

That’s unfair.

You certainly don't need rote for that, the algorithm is trivial for anyone who did well in their introduction to programming class.

Either the candidate "gets it" in about 10-15 minutes, or struggles for 60 minutes.

The pressure of whiteboard interviews (mostly negative; of the form "yeah this problem is basically trivial, but I'm not used to coding while talking to a stranger, plus the time requirements just don't smell right and certainly do not reflect real work conditions") are so infinitely removed from ...

The pressure at real jobs (mostly positive: "the problem is somewhat non-trivial, and I know time is kind of tight; but I'm in the flow and in my preferred environment; and at least I know the requirements are legit; so can I triage some good result out of this before the end of the day?").

At least for any job one would want to have.

If this distinction is not intuitively obvious to you - really, you have no business constructing interview challenges for people. Not challenges with any kind of a time limit, anyway (these being entirely unnecessary, when you think about it).

Are the best developer candidates looking for on-site roles? I don’t know any that prefer an office setting.

2. Whiteboards

Yet another way to test developers on stuff that is unrelated to the work. I don’t even like writing with my hands all that much because I’m on a keyboard all day.

I get into a whiteboard and I am being tested on writing skills, how visual my thought process is, how my handwriting is, how comfortable I am in a public speaking position…all great qualities, but all entirely unrelated to programming in a team.

Is whiteboarding even a realistic exercise in typical multi-office or remote companies? If I’m presenting to coworkers I’m probably on zoom in my development tools right? That’s not a whiteboard.

all of these are _highly_ relevant to programming in a team beyond the senior level

so, people want to hire me when they meet me in person, and I can give more inputs of rapport. data point of 1.

Few months later I got a call asking to interview again as they realized their standards were too high. Not a chance.

They could simply be wrong. I've seen people demand the wrong solution. I've seen tests with bad instructions.

Most people are not good at producing test

I don't know, perhaps it's because I've progressed every interview I've had the opportunity to do a take-home on (and I struggle with most leet-code style whiteboarding interviews) but I've always seen take-homes as an audition for your software development style -- what you deliver in a take-home assessment is what they will use to judge how you'll deliver on the job, plain and simple.

I've only done take homes for companies that I've already passed initial interviews for and generally like the people I've talked to and have favorable impressions of the culture/workplace thus far. I wouldn't do a take-home before that, but I'd do a take-home any day before a leet code grinding thing.

So your experience has been quite specific. A lot of companies think that it's fine to send you a take-home assignment as the first step. Some more do it after just a 10 minute phone call.

I'm not saying you're wrong, just that your amazement can easily be explained by the fact that you've done it in particular circumstances which is probably not the experience of most people.

Do you also expect a Med graduate to see some patients in the hospital for free before giving them a job?

I've seen individuals that can't speak English, can't spell, use punctuation and write a coherent sentence given a degree. I don't want to bring race into this, but it's very much an important issue here in South African universities: there is a huge shortage of black developers. So universities have immense pressure to rubber-stamp pass the black individuals lest they be seen as the "problem" or "racist".

If you spend enough time hiring you learn that a 4 year degree is a very poor signal for "can be a productive engineer". Communication, raw coding, and soft skills aside I'm still regularly shocked by how weak the CS skills of many fresh grads are, it seems a double digit percentage never really learn big O well enough to use it day to day. When you're paying big salaries you can't afford not to verify this stuff.

> Do you also expect a Med graduate to see some patients in the hospital for free before giving them a job?

I mean, this is literally a requirement for graduating most pre-med programs.

Getting more at your point - in many skilled white collar professions you need to take regular tests to maintain "certification". The bodies that issue these certifications are trusted enough to maintain standards that it's possible to get hired by maintaining the certification. SWEs have no similar institution, and given how fast the field moves it's not clear we're ready to establish one.

If the computer science industry was more well regulated along the lines of traditional engineering by requiring a PE exam, then we'd have to do less upfront whiteboarding and leetcode interviewing to verify that an applicants abilities actually reflect their academic credentials.

A nice way to say this is, "I'd be happy to complete this takehome assessment, but it will take a significant amount of time and first I would like a short phone call to ask some questions and get a feel for the company, nothing too long, I just want to make sure we're all on the same page before I spend my time on this. When can we schedule a call?"

> I would like a short phone call to ask some questions and get a feel for the company

I've had some places do behavioral first and others do coding first. I think I'm more onboard with behavioral first because communication a lot of times is more important than raw technical skill. Everyone can save time if you can suss out early on that the person is a bad communicator or bad fit. And generally you can spend time in this interview asking questions about the company.

It seems most places do this though.

Twice I've aced these challenge with time to spare and was then rejected or ghosted.

It's the same idea; don't invest time into a company's interview loop unless you know they're investing something too.

I had this experience a few weeks ago with Wolverine Trading.

Who knows, maybe they're in a position where they can get away with pissing on candidates. Fortunately I'm not that desperate at the moment.

As others have mentioned, I find the "limit yourself to 2-4 hours" BS. I can stand up a new http server in 2 hours to implement some API, sure. Adding unit tests, integration tests, a readme, build instructions/scripts, containerizing it, adding detailed comments, linting... generally not possible to do in 4 hours unless you have everything set up before then (I can't use my normal dev tools to do takehomes because they're through my employer). IRL you would generally scope this as a week-long task even for experienced employees.

Maybe if I were unemployed and desperate for a job I'd suffer it, but most places I'm interested in working at ask whiteboard/design problems anyway.

Anyways, just show me the code and we'll discuss it together, don't waste my time making improvement to toy code that will never get used.

In my experience as an interviewer fishing for specific answers is what makes you miss the best engineers.

I used to do this during my peak Dunning-Kruger phase and gave relatively poor marks to an guy who later proved to surpass both me and the rest of the team in every aspect that mattered - even though originally he barely made it through the interview process.

Chiefly, he didn't do things the way we expected because he prioritized.

Are they always worth fixing, regardless how many there are?

Also, you had plenty of time to prepare these and think this through. Someone seeing this for the first time may assume Chesterton's Fence and not change anything despite noticing - assuming it's less important than breaking whatever reason there might have been for doing things this way.

What I'm getting at is that you're not necessarily getting the best engineers, just ones who think like the current you.

It's a mistake and it's likely already costing you.

I gave out one that was sort roman numbers.

I gave out a file that was 1000 lines long. It may contain duplicates. The numbers are all valid, but are not necessarily reduced (e.g. the number 4 may be represented as IV or IIII).

Print out the reduced numbers in sorted order. If both IV and IIII exist in the file, then IV should be printed out twice (in the proper spot of the overall sorted order).

I timed myself doing this from scratch. It took about 1 hour (7:38 pm to 8:48 pm with some thinking and cleaning sporadically afterwards - https://github.com/shagie/RomanSort/commits/master its 6c79654 that has it acceptably working)

One candidate did it very similar to what I had.

Another candidate didn't have runnable code. Because it was only to take about an hour, and their interview was Monday at 10 am, they didn't start it until they got into their place of current employment that morning (they had a full week to work on it). Neither I nor my manager was impressed with that one.

Another candidate put them into an array (because arrays were faster than lists to access), and then wrote a Comparator to take two roman numeral strings and called Arrays.parallelSort() (because that was faster too), and took the String representation of the array and did a replaceAll on the delimiter with a new line and printed that out. It worked, but the code was messy.

There's a couple obvious tests (written unit tests) that I would expect the candidates to think of for their roman number sorting function, and I usually graded the take homes better when there was 1) evidence of testing 2) evidence of smart, appropriate edge case testing, but not having time to write polished tests in 1h isn't necessarily a red flag.

I'm really struggling to find how simple examples like this can demonstrate anything like GP mentioned.

There is a risk here for something that even really strong intermediate programmers fall into where they take a very prescriptive approach to programming so they tend to not understand things built from first principles and also tend to overvalue things like future-proofing and abstraction. Since you don't mention any specifics it's easy for people to fill in the blanks here and assume that at least a good portion of the things you refer to are those kinds of things.

I have had the exact same experience. They have always been a complete waste of time for me and I have since adopted a policy that any takehome challenges = automatic close the loop for me. It's not worth explaining why either. If this is your process and you find it appropriate to offer it as a first option, then it's just not a good fit.

I wish people were less surprised when we tell them they deserve to be paid for their work, but companies have been advantaged in the hiring process for so long (excepting COVID) that disregard for people’s time is embedded in corporate culture.

However, to the point people have made about not getting comprehensive feedback: it’s just not worth it. Corporate counsel will basically refuse to let you tell a candidate why they were rejected for fear of any inadvertent lawsuit exposure. This is also deeply embedded in corporate culture and it’s hard to imagine it changing.

then why are you interviewing?

Besides, I don’t have to prove my competence with every client. The mere fact that I have an email address that ends with @amazon.com automatically gives me credibility.

if your first comm with me is a link to a test. i'm not going to waste my time.

it's a non-starter.

Not joking. You hated interviews before? Get ready to rock.