- No - Yes - You have to put other things on hold and come up with something non-stop code worthy, which may be frustrating.
There's a solution I like though - work on a known bug or new feature in some open source project, potentially of the candidate's choosing, but preferably not one either the candidate or interviewer is already a major contributor to. That way both people come to the problem with a similar low level of expertise and have to work together to come up to speed (which is a great real-work-like exercise). It also makes it obvious what to work non-stop on for that time. Bonus points for being way more useful than a make-work wheel reinvention project and for not being free work for the interviewer's company.
One piece of logistical advice here: keep different kinds of keyboards/IDEs available. I use a mac and I interviewed a candidate who wasnt comfortable with the mac keyboard. In an interview situation where a person is already slightly nervous the last thing you want is for them to be worrying about keyboard shortcuts!
The purpose of pairing in this context is not to see a meaningful contribution; rather, it's to see how the candidate interacts with a codebase he's never seen before, and developers he's never worked with before. Can they spot mistakes? Recognize patterns? Read and write code? Does the candidate know the ecosystem well enough to suggest methods or libraries?
Most importantly, pairing tells us whether or not the candidate can communicate effectively. Are they terrified to offer suggestions, or overly arrogant and controlling? Can they explain clearly? Do they clarify when the other half of the pair doesn't understand?
For example, many moons ago, I did some pairing with a candidate that gave me a suggestion on how I could make a section of my code both smaller and easier to read by combining a couple of iterators in an obvious way that I hadn't noticed. We ended up hiring that guy.
The getting-up-to-speed part happens as they pair, with a lot of explanation happening as they work on the system, and it's another important part of seeing how they work -- can the candidate understand new concepts quickly? Do they stop and ask questions when they don't understand?
This type of interview is a performance hit for that employee for the day, but this is fine, as I consider hiring to be very important, and losing a day or two of engineering time over the course of a month is a price I'm willing to pay -- keep in mind that the phone screens eliminate a lot of people.
Admittedly, I've never had a day when my guys didn't have work to do, but not all interviews involved the company codebase.
I make it a point to carve space out for my team to do miniature research projects, in a fashion similar to the way HP used to work, or Google's "20% time". These are timeboxed, and at the end, the result needs to be made available to the team; and yes, "this didn't work because of X and Y" is totally acceptable. This always pays off in terms of education, and every now and then you get something useful out of them, but they're also fair game for pairing during programming interviews.
Hopefully I've answered your questions.
- Are the candidates able to help make meaningful contributions to whatever the senior engineers are working on without having the background and domain knowledge?
- Do the employees have to spend a significant amount of the 2-3 hours with the candidate bringing them up to speed on what they are working on, just so that the candidate isn't totally in the dark? If yes, does this totally sideline the employees day/flow?
- What do you do if the employees don't have several hours of non-stop coding lined up for the day?
I don't doubt this method works well - but I'm curious how you work through some of the stickier issues of pairing a total outsider with a fully-up-to-speed engineer and then evaluating the outsider based on what was produced in those 2-3 hours.