1. A very loose time limit. Maybe, as a rule of thumb, at least 10x the time you expect the challenge to take (so, a 1-hour challenge is given a 10-hour limit to turn it in, and a 5-hour challenge is given two days). If you expect it to be done in real time, why they heck don't you just bring the candidate in for a normal interview?
2. If the challenge is expected to take more than, say, 5 hours, the candidate should be paid for their time. Long challenges are going to disincentive candidates with lots of good alternatives to your company, so compensating them as in your own interest if you want good engineers to make it through your process. Plus, it's a strong signal that you value their time.
3. The challenge should be as close as possible to the sort of thing you do day-to-day. This should be obvious, but don't ask math questions or graph traversal questions unless this sort of thing comes up regularly in the job you're hiring for.
4. The candidate should be given an opportunity to discuss their solution. In real-life code, a seemingly stupid decision is may well be totally justifiable, but not obviously so from the code itself.
5. The candidate always gets an explanation of why they were rejected. At one company I worked at, I was in the position of reviewing takehome tests (which were one component of a longer process which involved some on-site coding). I had to write those explanations, and I know it's hard. It's easy to say "your skills are not a good fit at this time", but it's hard to tell someone you don't know what's wrong with something they put a lot of work into. It's still worth doing.
Please name these idiots.
1) I didn't score well on the test. It's within reason they read this post and release my scores with a 'it was because he scored low'. Future employers then come across my scores and deny me.
2) Don't bite the hand that's like the hand that MIGHT feed you. You don't yell and storm out of investor meetings. Because then they tell other investors. Similar logic here.
3) This place is very optimistic. Pessimistic people aren't particularly desirable. Whistle blowers are included in the latter group especially by corporate entities. Who knows what it could cause.
It really boils down to the fact that it's not a safe environment to do so. Or at least it doesn't feel so.
I think you're playing the game of life with your scared setting turned up too high. You might get better results to turn the bold up a bit over time as you gain confidence.
My take-homes made me fundamentally question my ability as a programmer. And I'm a damn good programmer. That is wrong on so many levels.
I can only pledge that if I'm ever in a position to hire developers (at any level) that I'll never force that kind of experience on a potential applicant.
I feel their is a major gap between junior and senior developers. And in this environment a junior developer has zero leverage with the companies he/she applies to. So they treat us like cattle.
It's like with every generation gap, their isn't much point blaming those who came before us. We can only do the best we can and make an effort not to replicate their mistakes. The next class(es) of software engineers have to improve and be better then the ones before them.
I'm preaching, but I hope you get what I mean.
There's a gap between knowing concepts and knowing how to apply those concepts. There's actually a certain (limited) amount of artistry to writing code. Hiring a programmer without seeing code is like hiring an Architect without ever having seen a house he's designed.
You don't need an 8-hour take-home coding exercise to determine whether or not someone is capable of writing working code.
Like FizzBuzz I think the take home worked well as another data point. It easily filtered out people who couldn't follow basic instructions, people who were clueless about Java even though their resume said different, and people who simply didn't care. The problem also gave us something to talk about when the candidate came in for the in person interview.
Overall the problem was dead simple for anyone we would actually want to hire.
The first one, they asked me to solve an incredibly complex math problem that I had no idea about so I struggled with it for 5 or so hours before giving up. Probably good that they didn't give me the job; if they were expecting me to know the level of math they were requesting then I would have had a pretty bad time. So I guess the test worked in that regard.
The second one interview at a different company, they gave me more moderate questions after a phone interview- mostly questions testing knowledge of a specific language. When I asked how long it would take, I was told "you have a deadline of 8 hours from when you download the zip file to when you turn in your code, expect it to take the entire 8 hours". That was probably the most stressful 8 hours of my life, way worse than any whtieboard interview would have been. I didn't look away from my computer once the whole time, did not let myself get up to eat, go to the bathroom, or anything. My girlfriend was in the room and tried to console me while I was doing it and stressing out and I ended up yelling at her to be quiet because I was trying to hard to concentrate. I finished the last problem about 10 minutes before the deadline, zipped the files up, and sent it to them. I never heard back.
Favorite interview I've ever had was a combo of a live coding session and a phone interview. They used an online tool that they gave me a link to that had syntax highlighting. The interviewer on the phone could see where I was typing and could highlight stuff etc. Think google docs, but for coding. They asked me to write some algorithms, then asked me to re-write them recursively, then asked a bunch of questions about the limitations of the algorithm, cases when it would be used, etc. It was fun, not stressful in any way, and the interviewer even taught me some things along the way! I feel like they got a really good picture of my thought processes and my personality so they could accurately judge if I would be a good fit on the team.
Paying somebody to do the take-home coding assignment and paying them would definitely help alleviate some of the stress, but I'm still not a fan of the format. You don't know how long it took the person to make their solutions, and you don't get a good idea for how they work, only what they can do (both are equally important, in my opinion).