I've had a very similar idea to this as well.

It all stemmed from one particular interview I had years ago. The interviewer presented me with a very simple problem, I solved it. He then stepped up the difficulty a bit. This kept happening and as it got harder he started acting more like a co-worker where we bounced ideas off of each other.

Granted he knew the solution, but the mere fact that he presented himself not as an interviewer judging my performance but as a co-worker helping to solve a shared problem made that one of my favorite interviews.

I’ve had two very similar experiences as well. They were back-to-back interviews and in both the interviewers started simple, then stepped up the difficulty. During the whole process the interviewers worked with me, not against me. Of course the questions became hard and they were clearly looking for a good answer, but the whole process did not feel antagonistic at all.

That does sound pretty good - what company was it at?

It was for Amazon. I will say I had 4 or 5 other interviews at Amazon that day and most of them weren't nearly as good, one was actually atrociously bad.

They may have changed their interview process at this point though, I only interviewed for them that one time and it was quite a few years ago.

You don't even need to randomly select a question to do something like this.

In the past I was hired on the back of simply sitting down with the team lead, and us pair programming to implement what he happened to be working on at the time.

That gave a good idea of how quickly I could start contributing, how easy I was to communicate with, whether I was going to be a snob about the existing code, whether I have relevant pre-existing knowledge about the technologies being used and whether I had any interesting insights to offer. There was also a more standard interview process accompanying this (which I doubt I excelled at) but the pair programming excercise gave a real world insight into exactly what I had to offer and what I was like to work with.

I once saw a company that interviewed like this. The also paid the candidate for the afternoon. The thinking was if you are working on our codebase you should be compensated.

For the record, half of the technical part of Canonical's engineering interview track is collaborative; multiple candidates join in to troubleshoot in real time a system that is malfunctioning. We gain from that a lot of insight into how people work together and what strengths they bring to a team. I haven't seen something like this used anywhere else.

As an interviewer I'm going to give this method a try. I will also add a step to our hiring process where we solicit opinions from the team about how well they felt the interviewee did at communicating during the exercise, as this is a major part of any real world work.

I don't think the fact that the interviewer choose the same quiz all the time is the culprit (on the contrary i would say, because it helps establish a benchmark and understand the pitfalls of that particular question).

The problem is that getting the answer correctly shouldn't be the main goal of the quiz question. Rather, it is the opportunity to see how the candidate think, and how the person is able to work with someone else (aka the interviewer) to solve a given problem.

It could also be a way to make sure the candidate understands a few concepts of its field (complexity, memory pointers, etc).

Mainly what matters is the process, not the solution.

Back when a was a tech lead and made hiring decisions for my team I would run the questions I asked candidates by the rest of the team in a 1:1 setting. We were checking that people could answer the questions well -- and sometimes when they couldn't it could show us weak points and give us ideas for training.

I always ended those 1:1 sessions by asking if the team member would be comfortable working with people who couldn't get to the correct answer. And if they could, what they would want to see from a candidate working on the question.

I really like the idea of picking a problem together and working through it. I might try that in my next interview!