News for hackers. Stuff that matters.
Re:Hiring Without Whiteboards by leonidasv on Tuesday July 28, 2020 @10:19PM (#23982904)
I hate take-home projects more than I hate whiteboard interviews.

I think the best approach would be giving the candidate a broken app and have they debug it live or in a take-home debug-and-fix or debug-and-improve exercise. Could also have some refactor assignment.

Has any company tried something like this before?

Re:Hiring Without Whiteboards by underwater on Tuesday July 28, 2020 @11:29PM (#23983290)
I don't understand how take home interviews are being held up as a better solution. They have two major flaws:

They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.

There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.

As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.

Re:Hiring Without Whiteboards by koreth1 on Wednesday July 29, 2020 @12:36AM (#23983552)
My company does take-home interviews (or rather, we give candidates the option to do them; nearly all candidates choose take-home instead of a live coding session) and I don't think I agree with the "lowers the cost to zero" point. First of all, they are never the starting point; we only give a take-home problem to a candidate after we've had an initial discussion with them.

When we get a take-home submission, we essentially run it through our code review process, and as one of the engineers who does reviews, it usually takes me nearly as long to thoroughly examine a homework solution and write up my feedback for internal discussion as it would to do an in-person interview.

Being able to review a candidate's code on my own schedule rather than being yanked from the middle of some knotty problem to go do an interview at a fixed time is a big quality-of-life win for me. But the rate limit isn't much different than it would be with regular interviews.

All that said, it's totally possible our process is super atypical.

Re:Hiring Without Whiteboards by WalterBright on Wednesday July 29, 2020 @12:05AM (#23983428)
At Caltech, most exams were take home with a time limit of 2 hours. Some were "infinite time". The students hated the latter, because it means there was no end to the time you needed to spend on the exam, meanwhile there were other exams, etc., to take.

With a 2 hour exam, there's an end to it, and you know how much time it should take to do the problems. Much better.

It's even worse if you're being graded on a curve, as you would be in a job interview. You're competing against others who will spend infinite time on it.

Re:Hiring Without Whiteboards by azangru on Wednesday July 29, 2020 @08:50AM (#23986052)
Like you said, "there were other exams, etc., to take.". Presumably other competitors have other exams as well? You'll thus be competing against people with different time management skills.
Re:Hiring Without Whiteboards by azangru on Wednesday July 29, 2020 @08:48AM (#23986035)
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable

> I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.

Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?

Re:Hiring Without Whiteboards by jnwatson on Wednesday July 29, 2020 @12:07PM (#23988327)
You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.
Re:Hiring Without Whiteboards by azangru on Wednesday July 29, 2020 @12:20PM (#23988488)
- If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you?

- Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like?

I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?

Re:Hiring Without Whiteboards by jnwatson on Wednesday July 29, 2020 @12:57PM (#23988949)
The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience.

Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know.

In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).

Re:Hiring Without Whiteboards by azangru on Wednesday July 29, 2020 @01:28PM (#23989345)
> but bias against other desirable candidate qualities such as experience

Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof?

There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on.

I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.

Re:Hiring Without Whiteboards by solidasparagus on Wednesday July 29, 2020 @02:40AM (#23984022)
Totally agree. I've almost never completed a take home project (even though I liked the concept) because there were always other companies I liked that didn't demand as much of my time whose interview process moved faster and who gave me offers before I finished the take home interview process.
Re:Hiring Without Whiteboards by Jare on Wednesday July 29, 2020 @08:26AM (#23985844)
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable

I'm going totally nuts about this particular problem. I have written in as clear terms as possible that I really don't want the candidate to spend more than 2 hours, but I don't feel I am going to convince anybody - especially as this is a (mini)game programming test, and with games you can always add one or ten more things.

Re:Hiring Without Whiteboards by sethhochberg on Tuesday July 28, 2020 @10:45PM (#23983059)
We do this at String and Key (and I adopted the process from my prior teams at Digitally Imported. I've posted jobs from both here on HN in the past).

Its a one-hour hands-on programing lab via screenshare - a pretty straightforward task of consuming some data from an API, processing it a little, and displaying it. We use Ruby for our work, and ask the same of our candidates in the lab. No handwritten syntax, exotic/tricky algorithms, or gotchas. Just a task to see how a candidate approaches a problem, deals with requirements, and implements them in our language of choice.

Happy to go into more detail if anyone is trying to implement something similar in their own hiring processes.

Re:Hiring Without Whiteboards by leonidasv on Tuesday July 28, 2020 @11:02PM (#23983149)
I'm interested in hearing more. I'm currently in the process of improving my interview approaches — currently a live coding session with a real-world problem involving an internal dataset.
Re:Hiring Without Whiteboards by sethhochberg on Wednesday July 29, 2020 @12:40PM (#23988729)
It sounds like you're on the right track. I think the important parts are:

1) It should be a live exercise (at least mostly live). I'm less interested in the end result than how someone got there, and most people chose to narrate their thought process as they work through the exercise.

2) The candidate should be able to work in their own environment. Online assessment tools can be different enough to introduce friction in a timed interview scenario. I'm interested in seeing how fluid a candidate is with the tools they'll use every day.

3) It should have the same expectations as your day-to-day work. Did the editor have a linter configured, and did they use it? Were tests written for the parts of the lab that introduced nontrivial logic? Is there a README? Is it all one git commit, or did they habitually make commits as they worked?

4) The design of the lab should have a decision point - I'm currently using a lab which could be implemented successfully with either a relational database or some kind of cache store. The strongest candidates recognize there are multiple paths and can justify why they chose the one they did.

This works best for reasonably experienced devs who can actually be expected to create something from scratch from a set of requirements on a short time frame. For more junior people, I'd typically give a version of this lab that I completed myself and introduced a couple of bugs into. The task is to fix the bugs.

Re:Hiring Without Whiteboards by Aqua_Geek on Wednesday July 29, 2020 @12:03AM (#23983420)
Apple did this in one of the sessions last time I interviewed there. They handed me a project in Xcode that had a number of bugs with varying degrees of difficulty — some caused crashes, others were UI glitches, etc. The assignment was to identify the problems and fix them. If you have more than a couple years’ experience the issues were pretty straightforward.

I really enjoyed this session, but I can see how it could be stressful to others.

Re:Hiring Without Whiteboards by twic on Wednesday July 29, 2020 @01:32PM (#23989404)
We used to do this at an old job. We were a consultancy which specialised in a particular e-commerce framework, and were hiring people with experience in it.

So we set up a small project using this framework, and mildly wrecked it. Made some really basic mistakes, made some more subtle mistakes, added a bunch of dead code, wrote good tests for some of the broken stuff, wrote broken tests for some of the good stuff, etc.

Candidates came in, and we asked them to add some simple feature to the app, which we warned them may be broken. We had someone sort-of-pair with them as they work, to watch what they do and keep them on the rails to an extent. We then evaluated whether what the candidate did was sensible: did they run the tests, did they pay attention to the failures, did they find and fix the mistakes, did they get distracted by the dead code, etc. Candidates didn't have to do everything perfectly, but there is plenty of scope to go really wrong. I remember one case where the very first thing the candidate did was delete all the tests. Thanks for coming in, we'll let you know!

The company did quite a lot of rescue missions, where we would pick up a project that another consultancy had started and failed to deliver, and get it live. This interview wasn't just a trick, it was testing everyday skills!

Re:Hiring Without Whiteboards by barrenko on Wednesday July 29, 2020 @02:05AM (#23983880)
This sounds really cool.
Re:Hiring Without Whiteboards by bhntr3 on Wednesday July 29, 2020 @08:10AM (#23985734)
> I think the best approach would be giving the candidate a broken app and have they debug it live

I had a company do this and it went really poorly for me. It was a python app and at the time I wasn't very expert in python. They told me to use my laptop and all I had for python was a vim environment. I knew enough python to do white-boarding problems. But debugging a complex app exercises a whole different set of skills I just hadn't developed in the language. I was also learning vim on the side. I looked like an incompetent amateur.

So there's a language and tooling issue. The bigger problem is that debugging an unfamiliar code base under time pressure is almost as niche as doing algorithm problems under time pressure. I've been on enough company wide on calls that I actually think I'm pretty good at debugging unfamiliar code under time pressure. But if I have someone who understands the code looking over my shoulder, they're usually at least trying to help.

It's still time pressure. It's still a somewhat contrived problem and code base. The interviewer is still comfortably omniscient. And it introduces some more complexity due to unfamiliarity of the language, tooling, code base or problem space.

Ultimately, I'm not sure it's any better than white-boarding.

A lot of these problems are solved with a take home question but then all the problems with take homes rise in their place.

Re:Hiring Without Whiteboards by jb775 on Tuesday July 28, 2020 @11:07PM (#23983176)
Take-home projects are frustrating when they basically just say "Build an app that does $xyz", and we'll be in touch if we like it.

Obviously no one is going to submit an app that doesn't accomplish the primary task. I feel like you get judged more-so on how you decided to get the initial app up and running, which likely isn't something you'll need to do working there.

Re:Hiring Without Whiteboards by koreth1 on Wednesday July 29, 2020 @12:42AM (#23983574)
> Obviously no one is going to submit an app that doesn't accomplish the primary task.

I would have guessed the same thing before my company started doing homework interviews. But in fact, the ones that don't get to the finish line are sometimes the most interesting ones to review, and they aren't an automatic no-hire. (And we say so explicitly, if a bit more diplomatically, in our instructions.) If someone hands us clean, well-thought-out code and it's clear that they honored our time limit and just didn't quite get there before the clock ran out, we'll still talk to them.

Re:Hiring Without Whiteboards by toadi on Tuesday July 28, 2020 @11:02PM (#23983151)
We are a remote company. I give take home projects too. Mostly I give them couple of days but mention they shouldn't work more then 4 hours on it.

Our test is actually quite simple and everyone that did any kind of development should have done these basic things.

But it tells me a log of the current state of the candidate. Most code samples on internet are not production ready code. Security and error handling are mostly not included. So this is where a candidate can shine... but I'm happy if you can string it together working even if it is by googling.

Re:Hiring Without Whiteboards by leonidasv on Wednesday July 29, 2020 @10:17AM (#23986901)
> they shouldn't work more then 4 hours on it

So you're saying spending 3 and a half hours on the assignment is fine. It's not.

Take-home assignments have little to no value for the candidate, they're only feasible if you are jobless or in college. The last thing I want to do in my spare time is to stay focusing on a bogus problem that may end up discarded.

Re:Hiring Without Whiteboards by cutemonster on Friday July 31, 2020 @12:56AM (#24005802)
What would you like to do instead?

Talking on-site or on the phone? Is that better spent time

Re:Hiring Without Whiteboards by Ozzie_osman on Tuesday July 28, 2020 @10:21PM (#23982924)
Yes. I worked at Quora and we had a "practical interview" that was exactly this (it wasn't take-home though).

https://www.quora.com/What-is-it-like-having-a-practical-int...

Re:Hiring Without Whiteboards by zerr on Wednesday July 29, 2020 @05:44AM (#23984982)
Paid (premium rate) take home tasks is the way to go, after the initial screening.
Re:Hiring Without Whiteboards by collyw on Wednesday July 29, 2020 @04:09AM (#23984416)
Take homes are too time consuming.