It's a bit of an art to come up with tasks that let you accurately rank candidates.
It shouldn't take more than 30-45 mins for a competent programmer. There's also some instructions in the mini-design which are intentionally 'the wrong way' to do the task. This is to encourage the candidate to get in touch.
I think this shows their coding style, problem solving, but also how they can work with others. Junior devs tend to get in touch about the complexity of the problem, senior devs get in touch about the methods in the design.
This process hasn't yet let me down.
I sent the manager an email telling him that I couldn't write a K-D tree on my own, but I would use the implementations that I found online to solve the problem. He replied that what he was looking for was someone who would've relied on an existing solution to solve the problem instead of trying to code it all from scratch. So I passed his test and was called in for an in person interview which was just trying to see if I would fit in with the team. It all worked great and I was hired. I didn't have to do anything with a whiteboard.
At the end of the day I care about whether or not you can solve a problem.
I really don't care if you solved it because you can use google or because you have a network of people to ask questions.
Technology changes too fast to expect people to always know it all. I'd rather have people that are resourceful and can dig their way out of a hole.
The effect is not limited to physical presence, it can also happen over a phone interview with screensharing. Consider all these thoughts that a candidate is probably thinking while trying to solve the problem:
What's this person thinking? Am I working too slowly? Should I type code faster? Is this solution what he/she is looking for? Should I talk more? I'm not sure what to say, ok stop panicking, just focus on the problem. Should I say I'm focusing on the problem?
The problem is not the whiteboard. The problem is that there is someone constantly looking at the whiteboard.
The solution is simple and I'm surprised that I've never seen a company try it.
Give the candidate a big whiteboard or a laptop with no network connection. Give a problem and leave the room. Come back later. When depends on the size and scale of the problem. Put the code on a projector and get other engineers in the room. Now discuss the code. Run the code through some sample data. Make some criticisms and see how the candidate defends their decisions. It's like a long code review session. Digressions are ok. Maybe talking some data persistence will lead to a discussion about caching. Maybe asking why the candidate did something will lead to talking about concurrency.
A bad candidate can't BS through this. They wrote the code, they have to either explain or defend it. And the way they talk about things also gives a chance to show their level of experience. The way a junior dev talks and a senior dev talks is very different.
The key thing is, let the candidate have some time alone to work. Stop pressuring them with having an engineer there constantly. The engineer doesn't like it because they would rather be working on something. The candidate doesn't like it because it's stressful. And there's no benefit to having someone there constantly. You don't need to hear their thought process in real time. You don't need to see the iterations they went through before the final solution. If you really want to know, you can find out all these things in the discussion session afterwards.