News for hackers. Stuff that matters.
Re:My Favorite Engineering Interview Question by lrm242 on Wednesday October 27, 2010 @11:52PM (#1841646)
These types of questions are terrible interviewing techniques. An interviewer who, even jokingly, goes into the room with an interrogative mindset is fundamentally failing at the core purpose of interviewing: finding good talent, preferably relatively undervalued talent. An interviewer should not be looking for someone who mirrors their way of thinking or or their approach to solving a problem. They should be looking for people with skills that complement the team, not duplicate it. Questions like these are typically signs of bad interviewing technique, but not always depending on how they are presented.

More often than not, however, people "design" interview questions to cause problems for prospective employees. You might think this is clever. You might think this helps weed out people who don't possess the ability to think quickly or react to feedback on their feet. All you really are doing is artificially limiting the prospective candidate pool.

Every dork with a chip on their shoulder can manufacture an a question to be unanswerable. Deep down inside what most of these bad interviewers really want is to design a question that creates one of two atmospheres in the interview room: (a) a submissive one, where the candidate is forced to placate the interviewer and reinforce their apparent intellectual superiority; or, (b) a confrontational one where the candidate and interviewer battle it out for dominance as alpha-geek. Either (a) or (b) can be seen as a positive depending on the type of interviewer that goes in. Ultimately, any interviewer that uses this technique is a bad one and any input they make into the hiring process should be seriously discounted.

The interviewer should adapt their interview style to the candidate based on reading their body language, communication skills, and overall demeanor in the interview room. Its the interviewer's responsibility to create the best interview experience possible. Some people are nervous; some don't think on their feet well; most rely on Google heavily to point them in the right direction; lots of them know more than they can demonstrate in the 5 minutes you give them to react to your stupid question.

Your job is find diamonds in the rough. People who are the best, not people who act, think, behave, or answer the way you think they should. Questions like the one in this blog post are tell-tale signs of alpha-geek interview dominance gone wild, and out of that you'll typically only get one style of candidate that makes it through the gauntlet, certainly guaranteeing you're not getting the best talent available.

Re:My Favorite Engineering Interview Question by jemfinch on Thursday October 28, 2010 @02:03AM (#1841939)
I don't understand your objection here.

The question presented is a simple, open-ended question with a number of acceptable solutions. It's not even remotely "unanswerable". It's at least mildly interesting as indicated by the number of responses at a similar question on StackOverflow: http://stackoverflow.com/questions/2573653/given-a-1tb-data-... . It's not far from the problems Ning solves. It's not an "aha" problem that you either get or you don't get. It's complex enough to reveal the candidate's thought process and problem solving methodology, but it's small enough for good candidates to get to a reasonable solution in the 45-60 minutes of an ordinary interview.

Nothing about the article indicates that the author has a chip on his shoulder, nor that he's a dork who wants to "battle it out for dominance as alpha-geek."

Nothing about the article indicates that author is looking for a specific solution reflective only of his particular prejudices in programming.

What in the world is the basis for your entire diatribe here?

Re:My Favorite Engineering Interview Question by lrm242 on Thursday October 28, 2010 @07:56AM (#1842533)
Seeing as you worked at Ning and likely participated in these types of interviews it's not surprising you don't get it. Surely Ning was a great place to work and I'm sure you found really top-notch employees through your well known tough hiring process, but I'd be willing to wager they all look very much the same when you blur your eyes a little bit.

My diatribe, as you so eloquently put it, is based on the article. The list of things that the author expects to see in the answer. The way the question is presented. The lead off with "interrogation". The "gotcha" follow-up regarding how things might fail. These questions are great at finding a very specific type of candidate, but the entire point of my comment is that this one candidate type is typically not indicative of the best engineer. Of course you'll disagree, I'm sure, seeing as you were part of this process.

Re:My Favorite Engineering Interview Question by brianm on Thursday October 28, 2010 @11:07AM (#1843140)
Asking how it will fail is a gotcha? To my knowledge and experience thus far, all systems fail. The question is "given this one, that we now have a good shared understanding of, which you designed with the tools you are most familiar with, how is it going to fail?

Not one candidate I can think of has been either surprised by this question, or felt stuck, fwiw. Again, I may be completely self delusional, but I hope not!

Re:My Favorite Engineering Interview Question by mquander on Thursday October 28, 2010 @11:00AM (#1843114)
Well, I'm eager to hear what you think the candidate type of the best engineer is, why that type of candidate would fail to come up with a reasonable train of thought given the cited question in an interview, and how you select for that candidate type.

If questions like the one posted find a type of candidate who can think well under mild pressure, who can quickly brainstorm sensible approaches to technical problems, and who can communicate their ideas and reasoning to their peers, then that sounds like a pretty good outcome to me.

Re:My Favorite Engineering Interview Question by jemfinch on Thursday October 28, 2010 @12:29PM (#1843549)
Wow. You seem angry, or frustrated. Did I say something that offended you?

I only participated in Ning's interviews as an interviewee. I worked for Ning for six weeks before the layoffs.

The only thing I'd notice when I blur my eyes a little bit is that Ning employees were better engineers than most I'd previously worked with. The same applies at my current employer, also known for its rigorous interviews. The similarities ended there.

You complain here about the things the author expects to see in an answer. He's looking to see what they think of that scale of problem and how their thoughts comport with their resume. He's looking to see what questions they ask to get more detail. He's looking to see that the solution is in reasonable bounds for hardware--that the candidate understands the limitations of the silicon we have to work with. He's looking to see whether they would reinvent the wheel or use proven technologies. He's looking to see what languages and technologies a candidate would choose to use. He's looking to see how well the candidate understands those tools. He's looking to see how cognizant candidates are of the failure modes of the system they're designing.

Maybe I'm dense, but I don't see any issue with the list of things the author expects to see expects to see. Can you be more specific, perhaps?

Likewise I don't understand the problem with the way the question is presented. He describes the data you have and the load you need to support. Is there something wrong with that presentation?

The crossed-out "interrogation" is obviously a joke. I'm certainly not known for my intuitive perceptiveness and I still saw that (and no, I never met brianm while working at Ning, to my knowledge).

How is asking about failure modes a "gotcha"? Every good engineer needs to consider how his designs and systems can fail. What in that question do you expect trips up good engineers?

The only "specific type of candidate" this question seems oriented toward finding is "good engineers." He's not asking for particular technologies. He's not even looking for a particular solution. He's just asking people an open-ended question about how they would design a system. All the best engineers I know would be able to answer that question in 30 minutes.

I don't disagree because I was part of the process, I disagree because I can't understand your objections. I hate to sink to personal attacks like you have, but have you considered that your opinion on questions like this may have been formed by your abilities? It seems a more likely possibility than that questions like this are fundamentally flawed in some way that you've as yet been unable to describe. Both companies I've worked with that have had rigorous interview questions of this sort have hired the highest quality talent I've ever had the pleasure of working with.

Re:My Favorite Engineering Interview Question by bjnortier_hn on Thursday October 28, 2010 @06:53AM (#1842436)
I would say that having "interrogation techniques" crossed out is a pretty good indication that the author's state of mind in an interview situation is aggressive and domineering.
Re:My Favorite Engineering Interview Question by Goladus on Thursday October 28, 2010 @07:26AM (#1842488)
I would have said the same thing until I read the rest of the article. In context I think it's meant primarily to be funny.

The author clearly maintains some expectations about the level of response from the candidate, but overall it's designed to be interactive. Chances are that unless the candidate gives a really superior answer, it's not going to be the only question asked.

Re:My Favorite Engineering Interview Question by jholman on Thursday October 28, 2010 @03:45PM (#1844585)
What is the meaning, in an essay, of a cross-out word?

Generally, I would say that any time an author takes the extra trouble to write in an extra word and cross it out, they are making some kind of meta-comment deliberately, rather than straightforwardly revealing that the crossed-out word is representative of their true state of mind. The crossed-out word is a meta-comment on what the author thinks about the shared idea-space of the author-plus-audience, usually to say "hey I know that we all know that this concept exists", usually to also say "and that this concept is an unmentionable", and often to also say "and I think that's a bit funny".

In this case, we all know that some people conduct interviews as interrogations (and some people think that indeed they should do so), and on the other hand some people think that this is offensive. And brianm wants you to know that he knows we all know it.

MAYBE it's on his mind because he's an aggressive domineering jerk who likes to interrogate people, but he's sensitive enough to be a little defensive about it. MAYBE it's on his mind because he's NOT aggressive or domineering, and he's frustrated by how easily interviews become interrogations, and his whole deal is trying to prevent that without making the whole interview meaningless. MAYBE something else. How're you going to find out? Read the article!

My point, tl;dr: If you see a word crossed-out, by an author you don't already know, and you think "Oh this author really means the crossed-out thing", I think that's a poor comprehension strategy.

Re:My Favorite Engineering Interview Question by jemfinch on Thursday October 28, 2010 @12:00PM (#1843417)
I would say it's obviously a joke. Do you always assume the worst of people?
Re:My Favorite Engineering Interview Question by variety on Thursday October 28, 2010 @02:48PM (#1844322)
Maybe he thought he was making a joke, but the point is that it comes off "Ha, ha, only serious":

  http://sunsite.ualberta.ca/jargon/html/H/ha-ha-only-serious.html
Re:My Favorite Engineering Interview Question by scott_s on Thursday October 28, 2010 @02:55PM (#1844355)
It does at first, but upon reading the entire essay I was convinced his intentions are honest.
Re:My Favorite Engineering Interview Question by Goladus on Thursday October 28, 2010 @07:32AM (#1842495)
> Questions like these are typically signs of bad interviewing technique, but not always depending on how they are presented.

Given that the author presents the exact question and includes a moderately lengthy discussion on how he uses the question, what is your evaluation in this specific case, rather than the general one?

Consider, for example, this comment from the article:

I’m looking for quite a few things as we go through the question. The first is their opinion of “a terabyte of data” and “5000 lookups per second.” Do they consider this to be a lot of data, or a fairly boring amount, same with the lookups per second. Leaning either way isn’t a failure, it is just information gathering for me, and referencing it against how you represented yourself in your resume, cover letter, and phone screen.

Re:My Favorite Engineering Interview Question by broofa on Thursday October 28, 2010 @09:48AM (#1842834)
Your reaction says lot more about your attitude than it does about the OP's. I found the article interesting and informative, with none of the passive/aggressive overtunes you mention.

Your lip service about finding diamonds in the rough and catering to candidates who are nervous or can't think on their feet leads to just one comment: "Put up or shut up."

The OP has given us a very good, concrete example of his interview process, the traits it reveals in candidates, and how he tailors it to meet individual candidate reactions. Until you do the same your comment here is nothing but five paragraphs of whining about how some people don't interview well.

To which I reply, "Duh."

We all know the 45-60 minute interview is imperfect, and that both interviewer and candidate are just trying to make the best of a very contrived interaction to begin with. So, please, lose the attitude and give us something we can learn from rather than bashing the OP with vague generalities.

Re:My Favorite Engineering Interview Question by brianm on Thursday October 28, 2010 @11:01AM (#1843119)
I appreciate the feedback, but don't think I fall into, nor describe in the linked blog post, the interviewer category you describe (though it is certainly possible I do and just don't realize it).

Do you consider this question to be unanswerable? As I stated, I'll accept pretty much any viable answer, my goal is to get folk to design something ~realistic (okay, this is a simplistic system, but we have finite time in an interview slot).

Honestly, I think the question I blogged about is pretty straightforward, and exists for purposes of framing a discussion about real systems and constraints. There is no answer I was fishing for at any point I used this question.

"Questions like the one in this blog post are tell-tale signs of alpha-geek interview dominance gone wild," is so far off base from what I am trying to convey that I think I utterly failed to convey my point, which was quite literally the opposite :-(

Re:My Favorite Engineering Interview Question by variety on Thursday October 28, 2010 @02:39PM (#1844270)
Part of what's problematic about this interviewing approach is that while your (secret) internal mindset may be "I'll accept pretty much any viable answer", the full-court-press interviewing style effects the exact opposite impression on the candidate -- they're left thinking, "gosh, I have to get this right, dammit, no room for slack, can't leave out any details AT ALL or it's back to another month of ramen and job boards trolling."

So yes, with your interviewing technique (orthogonal to the technical content of the question itself) you've succeeded in conveying a stance and mindset that is quite literally the opposite of what you intended.

Re:My Favorite Engineering Interview Question by jholman on Thursday October 28, 2010 @03:48PM (#1844599)
Is "full-court press" and "your interviewing technique" from personal experience with the OP? Or just from reading the article?

Because all I got from the article is that he has lots and lots of follow-up directions. I.e., that he is prepared.

Re:My Favorite Engineering Interview Question by variety on Friday October 29, 2010 @01:22PM (#1848052)
From the article. The flip side of the fact that the interviewer has had time to carefully prepare while the interviewee has not is that what you get from that inherently contrived, lopsided discussion, kind of a long, drawn-out game of "gotcha." Fun for the <strike>interrogator</strike> interviewer, but tedious and buzz-killing for the interviewee.

It provides insight, yes, but a certain cost: many candidates fine this approach simply alienating, and more to the point, it upsets their thought processes (such that they don't really get a chance to shine).

As a candidate you can train for this kind of confrontational approach, of course, but I think lrm242 nailed what's most alienating about this interviewing style -- that ultimately the candidate is put into a position that's basically uncomfortable to be in, where they have to _placate_ their interlocuter according to some hidden criteria set -- and mostly they just want that particular thread to be overwith as soon as possible.

I just think we can get the same insight into candidates from different ways, where things are more on an even keel, and there's more "flow."[1] It takes more tact and creativity (and perhaps a bit more time), but it's not that hard, is more mutually enjoyable and (I think) ultimately provides a lot more insight.

[1] "flow" being of course a very important concept; e.g. in the sense of Murakami.

Re:My Favorite Engineering Interview Question by Retric on Thursday October 28, 2010 @11:53AM (#1843379)
Do you consider this question to be unanswerable?

Yes. Given a little time I could come up with 20 or more so ways of doing this. As to which one I would use that's not really answerable in an interview. The problem with questions like this is two weeks is a fairly long period of time. If it's literally all your working on then researching a few options and then asking some more questions is the best place to start. aka Is the access pattern random or would caching be useful? Is the data private? etc.

Re:My Favorite Engineering Interview Question by dbt on Thursday October 28, 2010 @02:23AM (#1841972)
I don't know about you, but when I am interviewing people I am not trying to find "diamonds in the rough", I'm trying to find the best person to join the team and contribute. That may or may not include experience, but it certainly includes the ability to solve problems.

This isn't an unanswerable question. I've asked and gotten great answers to a very similar question about handling data at scale. And what I'm looking for includes the opposite of a chip on the shoulder.

Re:My Favorite Engineering Interview Question by tlb on Thursday October 28, 2010 @02:33AM (#1841986)
Some interviewers are searching for talent, some are weeding out bozos. (The categories overlap: I know some talented bozos.)
Re:My Favorite Engineering Interview Question by palewery on Thursday October 28, 2010 @08:21AM (#1842577)
How do you evaluate problem solving skills of candidates during interviews?
Re:My Favorite Engineering Interview Question by seigenblues on Thursday October 28, 2010 @12:05AM (#1841676)
Man, I can't up vote this enough. I would also add that this is also usually symbolically equal to what VCs seem to do in their "interviewing"/wooing process. Well said!
Re:My Favorite Engineering Interview Question by garazy on Thursday October 28, 2010 @02:04AM (#1841942)
I couldn't agree more, when I read the "interrogate" introduction it put me off reading the rest of the article.
Re:My Favorite Engineering Interview Question by follower on Thursday October 28, 2010 @08:02AM (#1842547)
Did you read the rest of the article anyway? The "interrogate" wording may present a bad first impression but the remainder of the article is pretty reasonable.