If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.
(Even moreso if it's a fizzbuzz level task, I'd have thought - actual fizzbuzz being famous as a coding interview task, after all.)
Would you think doing this live in a screen sharing session adds more value? How would you prepare the candidate for this?
> Would you think doing this live in a screen sharing session adds more value?
In my experience, hugely yes. Above all else I really like that the time commitment is fixed on both sides - no homework for the candidate, and we're not asking them to invest any more time than we are.
Also doing things interactively gives a lot of leeway for adjusting and avoiding wasted time. If somebody's got one section basically solved, I might cut in and say "yeah that bit is great, let's call it finished and look at this other bit". If they're obviously having a lot of trouble, I might ask about their current work and maybe switch tasks if there's a better fit. Or if they're obviously coasting easily I might cut in with "how would you scale this?" questions that give more signal than watching them finish up the loose ends.
Another thing I like is that we can keep the task specification simple, like what you'd get from a PM, and it's up to the candidate whether they want to ask questions or jump in. I imagine that with take-home tasks you either need to give out pretty specific requirements, or else have people interpreting the task a variety of different ways.
> How would you prepare the candidate for this?
We explain at the beginning that the goal is write code for an hour and watch how they work, and there's no hard requirement for what needs to be finished by the end. And they're welcome to search the web, use AI, or anything else they'd typically do while working.
And I've only done a few dozen of these, but personally I've never seen anybody get particularly nervous. I think nerves are a bigger issue with HR style "pass all the tests within the time limit" tests, where the danger is that somebody can hit a wall and never get past it. That doesn't happen with my format, because if somebody is stuck I just give them hints until they're unstuck (since I get no signal from watching somebody scratch their head).
At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions).
If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?