Take homes can & ought to be timed.
I've had the opportunity to do this fornat twice and I think it works well.
Here's how it works: They ask you to pick a 3 hour time window to work on the project. You pick the time. They send you the project prompt in an email right at the beginning of your time interval. You have 3 hours to read the prompt, design, implement, test a solution. If you miss the deadline, project counts for nothing.
Both the times I did this the prompts were absolutely doable in 3 hours. They were the kind thing you could do in about 2 hours if you had the ability to read the project prompt ahead of time and think about it for a few days.
Personally, I work well with time constraints. This format feels closer to a workday where mid-afternoon you might challenge yourself to finish a task before 5pm. Its a lot less pressure than having to juggle a conversation and having your code scrutinized in real time.
It forces a real time constraint that keeps you from piling time into something you just don't know.
Its hard to cheat, and it also forces the company to make the task achievable in the time frame. This format forces them to realize when the task is too big for anyone to finish in the time.
My brain just not work that way. If I have to modify something in a project I already worked on. That's fine. But interviews always ask tho build something from the ground up.
Once I got the requirements I need a few hours to think about it, before I write any actual code. I usually do something else while I do this. On a job, it's when I home and do some chores and my mind can wander around freely. But I can't make this process much faster. If I'm on the clock I have to sit down and start spitting out code. Also, there is no time for any major refactoring if you went in a wrong direction. This adds a lot of unnecessary stress.
I like it when you have a day or an afternoon to finish a 3 hours long task. I work faster if there's no time pressure.
> I need a few hours to think...before I write any actual code...I work faster if there's no time pressure.
It sounds like you work better but not faster. If something absolutely must be done in 3 hours, you usually will not have 2 hours to do the dishes while you let it marinate.
To be clear, I prefer to let a problem sit in my head for a bit, too. I think the end result is better in most cases, for most people. But the real world has problems with hard and fast time constraints, or huge financial incentives to get something working in ASAP and clean it up later if necessary.
If something /brand new/ needs to be built in HOURS with huge financial incentives, that is a failure on so many levels of business that blaming the engineer for their inability to actually get it out the door in 3 hours is ridiculous
When I presented with a completely new problem I have to think about what is the best tool for the job, how I want to structure my code, how can I test it, what are the edge cases, etc.
I think I’d be fine if someone gave me 3 hours to complete a typical take home exercise, but I worry that some exceptional engineers I know might not be. Applying excessive time pressure could lead to cultural homogeneity.
Whoops, your home internet has gone down. Or Gmail has decided it's not having any mail from their domain. Or your mail provider think they're spam. Or your mail client doesn't pick up the email for half an hour. There's a whole raft of problems coming from "email delivery is reliable enough to be a timer."
The candidate should be encouraged to submit early and often for feedback, and to not be afraid to ask questions. Sometimes a subtle or complex element of the problem statement can help spur this interaction.
Finally the grading should be a blend of both objective criteria such as a series of heuristics and unit tests, as well as subjective measures such as code cleanliness and structure.
Multiple people should be involved in the grading, and a repository of previous submissions should be compared against for consistency.
I and my team used this technique for years to great success, hiring some of the world's top talent.
How do you judge someone's half-finished work? Do you expect me to write half-finished code on the job?
It's just an unnecessary source of stress during the test.
A take home interview therefore gives me a much better picture of the real-life skill level of a candidate compared to a very stressful situation like a real-time whiteboard interview.
Also: Give your candidate a real problem to solve and pay them for their time!
The whole interview was inspiring. It's about learning and exploration. Worth watching if you understand Hungarian: https://youtu.be/rtqhA4TtBrI
Strange reasoning. If you want to cut corners, you use stack overflow.
Unless you know by heart the optimal way on how to reverse a string in C#, you do a 3 second google search and do a 5 sec copy paste of the one liner, from the top answer on stack overflow.
When I was programming as a 14 year old and came across a problem, it sometimes took me a day to figure something out. There was no internet back then. Now the most obscure issue is probably nicely documented with a top 5 of solutions.
Also, when I interviewed a year ago I was asked to find if something is a palindrome. This is programming 101 and it caught me off-guard. I got so used to all the methods available in C#, that going back to pen/paper and writing a simple for loop felt odd.
1. Obviously, its faster. I, personally, get paid for results, not for how "clever" I am. (Which is good, 'cause I'm not very clever.)
2. Frequently, the answer on the internet is better/more complete/tested. Take implementing a Singleton class; its so simple that anyone could make one. Most home grown implementations would have race condition though.
One of the ideas that is always brought out about the awful interviewing process is that there are just a bunch of totally incompetent programmers out there that need to be kept out of because of fear of bringing quality precipitously down. I'm not sure how many this is and not just bad interviewees based on experiences, but I do also accept that they exist (My guess it's 10-20% of programmers).
At one project I was on we had a guy who was pretty bad (he was then hired away by a competitor to their super programmers team so whatever) but I know he spent days on something that should maybe have taken 1 day, and it looked pretty bad (I mean just looking at it)
I can say that no amount of time would have allowed him to write something that did not look messed up (based on his habit of using map to update multiple arrays at once and not using the returned array from map anywhere)
Maybe they've used extra hours to get a good result, but if they are able to recognize a good result they are not in that small group of programmers that must be kept out at all costs.
I suppose it depends how you define "quality" - in my recent experience, it's always been the "rockstars" that are the worst developers. Because they generally write code which ends up being a nightmare to support and maintain when they're gone. Sure, it's great you wrote your own ORM in Perl but 10 years later it'll be crippling the company because no-one can make any changes without it breaking 27 other things.
(Ok, it's often Perl people...)
For me, a quality developer is someone who achieves the goals of the company in understandable, maintainable, testable, and documented code.
Also he thought I had done something using ElasticSearch because I was an ElasticSearch guy, instead of having learned ElasticSearch because I thought it was what was needed (and no amount of explaining this simple concept could penetrate)
Agree. But it should be noted that the needs of a startup may not always make that easy. I'm personally responsible for transmogrifying an existing Perl ORM for a company, because it only supported a single database handle / class. Which is a pain when you're in a master / slave environment. This hack allowed the company to meet the performance challenges that came along each year by summer. It also allowed growing the company from ~35 people in 2004 to about 5000 people in 2012 (when I left).
It also froze the version of the ORM in question, until it got rewritten from scratch at great expense. But that expense could now be easily carried by the company.
Hiring heavily from colleges results in 60%+ of applicants that somehow have a B.Sc. from a university that produces amazing applicants with great (3.5+) GPA's but stall at either fizzbuzz or (just in case they memorized fizzbuzz) i-can't-believe-it's-not-fizzbuzz.
Furthermore, some candidates (20ish percent?) can't even answer basic, non-trivia questions about their language they self-report to know best. e.g. Java programmers who either don't know or cannot articulate what the static keyword does.
I'm convinced there's a path through every university, no matter how good, where you can avoid all the good professors and only pick babysitter professors and squeak through your four years with a piece of paper and a well-tuned sensor for which profs actually grade homework instead of just giving 100's so people don't complain.
But often when I am brought into a big organization there is one out of every 5 programmers that just should be something else. They're not necessarily stupid, just shouldn't program.
on edit: in my experience small organizations don't have quite as bad because for the organization to be able to exist their programmers must be able to program. (for any organization heavily focused on tech of course)
either that or 25 dollars, same as in town.
If you just evaluate them based on their code you doing it wrong. You should organize an in-person code review. Where you ask: Why they do what? How would they improve their code? If something is not right, point out. Look at them how they react. Good candidates usually try to fix it. Bad team players usually defend their implementation until the end. It really easy to find out if they just copy-pasted some code without understanding it.
- take home interviews incentivize the candidate to spend more time than they’re told. You can choose a three hour problem, but candidates that really want the job will probably spend more time. Less experienced candidates will spend more time on it too.
- I never found reviewing take home interviews to be a great indicator of skill level. It’s hard to know how long was really spent on it, and how much the candidate used resources etc to guide them. Are they a great candidate that cut corners, or a less experienced candidate that used stackoverflow + 10 extra hours to get to a final result.
- whiteboarding can go badly too. The caveat is that if it does, only so much of the candidate’s time is taken up; there’s a hard ceiling.
- from the perspective of conducting whiteboard interviews, the company loses out if the interviewer doesn’t do a good job. This is really on the company though - it’s worth investing in a good whiteboarding process. If there’s no investment here, it’s only the company that suffers (competent candidates will get hired by a company with a better process).
In terms of conducting a good whiteboard interview, my $0.02 is no trick questions, just solid problem solving with plenty of scope to show creativity and proficiency.