Like many others before me, I've discovered that there is a huge gap between the ability to talk intelligently and productively about solving programming problems, and being able to actually solve programming problems. It's fine to hire someone who can reason through and discuss solutions... as long as you're also going to hire someone who will write the code for them.
Otherwise: get sample code, and watch them code something.
and watch them code something
You may find http://www.interviewzen.com/ useful. It's a hiring tool I'm building around the idea that you can tell a lot about someone's coding ability by seeing them in action.Feel free to drop me a mail and we can chat further.
I have to have some "minimum" requirement questions. These are questions that you have to get right. No matter how good I feel about you, I still will no hire you if you get these questions wrong. These are questions that are so easy that most people on HN would scoff me even asking. Yet it actually does eliminate some people.
"You have a bunch of objects which have an implicit ordering. You want to insert them into a collection, look them up, and retrieve the smallest/biggest one as quickly as possible."
You'd be amazed at how many people suggest hash tables.
Count the number of occurrances of the letter 'a' in a string. And I'm upfront that this isn't meant to be a trick question having to do with locales or some odd corner of the Unicode world. If there's something funny about the question, let me know, I might learn something, is what I'd tell them.
I'd say 1 in 10 struggle with this question. The other 9 kick it out in 1 minutes. Very little time is lost in the interview.
Another example is to code the nth Fib number with recursion. I give them the recurrence relationship, in case they don't know it, and write some examples. I'm not throwing out Fib and asking for a solution, and secretly ready to pounce and say, "There's a closed form solution!" or what happens on integer overflow or anything. I want them to convert the recurrence relation to code as a recursive function. But I don't mind hearing valid concerns along the way, although really its just a mininmum bar test.
This eliminates about 15% who often don't get it working at all. (I'm really winging these numbers... i should have kept better track of this).
Doing well on these questions means nothing. It just gets you a ticket to the rest of the interview. But not being able to do them is an indication that I don't think I can listen to the rest of what you have to say (for a dev role) and be convinced that you can do the job -- no matter how versed you seem to be with the latest buzzwords.
Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.
http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-dev...
The interviewer started explaining what I'd be working on. In a few minutes it wasn't an interview anymore, but a free consulting session. He clearly forgot why we were there and even started to take notes!
I was offered the job, but later refused because another offer was better.
Additionally, this technique would probably work against college students and recent grads.
It can also be hard to quantify the results and the questions will vary wildly between candidates.
Of course none of those obstacles are insurmountable and I think that when this approach works, it probably works better than a more general tech interview.
I like your point about picking a problem you don't have the solution to. I'm guilty of making them solve something I recently did, and maybe it's not the best approach.
0. Explain to the candidate this algorithm and why. It isn't magic, there are no right or wrong answers, just testing "fit". This is a case where foreknowledge does not change the outcome!
1. Get a basic grasp of technologies both interviewer and candidate have in common, via say, just chatting. This is vital -- find an area where you are both somewhat competent.
2. State a problem you don't know the answer to, something you are currently working out for your own projects, in terms of the mutual tech. VERY IMPORTANT: you must not have the best solution in mind when you bring up the topic.
3. Brainstorm on it together.
This provides more feedback in a valuable way than the classic brainteaser. You find out how the person thinks and problem solves -- you know, that thing you are hiring her to do. You find out how you and her work together. Even if no solution is reached, it is OK because there was no solution going it -- just a problem to be worked.
Your job in the interview is then to make sure you steer the discussion towards various tech and deep problems, coding issues, etc. Afterwards you will have a pretty strong idea of how it is to work with the candidate, which is quite useful for deciding if you want to continue to work with the candidate as an employee. Further if you've done well at the steering of the discussion, you have learned about candidate's technical depth, without setting up test/interview anxiety situations.