None of those things are necessary to solve the problem. There's no need to search for information...all the information you need is contained in the problem. It's an exceedingly simple algorithm, and if you can't take the verbal explanation and translate it into code, you are simply not a programmer.
What would you search for, how to make a for-loop? If you don't know such basics as flow control and basic logic, you probably need to go back to school. If you can't remember the usage of printf then you either picked the wrong language, or ask for help, or simple explain you'll fudge it a little and a good interviewer wouldn't penalize you. I'd note if you are interviewing for a language-specific position and said you are fluent in it, not knowing such a basic API might count against you though...
I think the important thing you are missing is that generally in "whiteboard" type situations, you are not expected to produce a completely correct/compilable program. Generally pseudo-code or whatever language you please is accepted and if you make some small syntax or logic error it's not a big deal. In light of that, a compiler/IDE/computer is not necessary unless you have some other reason to require them...such as a disability.
The important part is to see how one reasons through the process of taking a described algorithm and translating it into code...that is the essence of programming. If you can't do it, you are ipso facto not a programmer.
Your analogy is false...in your example the person is not being tested on the actual job they are expected to perform. Obviously that's not a good test.
In this case the programmer is being tested on precisely what their day to day job is: taking a verbal description of a process and translating it into code....so an accurate analogy would be to ask an architect to sketch a simple truss structure and discuss it, which is not unreasonable.
The entire point here is to give a drop dead simple problem to weed out the people who are fundamentally unfit for the position.
Do you think this accurately reflects the working conditions of a developer? Do you think that they jump into problems and that small errors are OK to them? do you think that they will be comfortable making those small errors in front of people, given that they A) generally would not accept them for themselves, and B) how critical some people can be of code that is not their own? Do you think they will be able to put there best foot forward in such an unnatural situation? They may blank, they may freeze and they may just be the type of person that reflects on an issue for a while before the set out to code.
What would you search for
I personally would search for the mathematical formula and a text description that I could put on my second monitor as reference. But first I would search for a standard library and avoid the effort all together.
taking a verbal description of a process and translating it into code
I personally never work from verbal descriptions, I document the issue, and place it into a ticket system in which I work the tickets based on priority.
Do you think this accurately reflects the working conditions of a developer?
Who said anything about trying to simulate a normal development day? This is about finding out some of their thought processes and if they can solve an exceedingly simple coding problem.
They may blank, they may freeze and they may just be the type of person that reflects on an issue for a while before the set out to code.
Where did they say you can't reflect on the issue? That's exactly what they want is you reflecting on the problem and describing how you'd go about solving it.
If someone tells you to write a function that takes a number N and prints out 1..N in pseudo code(Which is basically what the pascal problem asks when they provide the formula for a row) and you can't even talk your way through a possible solution you're not going to be able to complete any interview process, no matter how gentle.
I personally would search for the mathematical formula and a text description that I could put on my second monitor as reference.
So you're complaining that you can't search for something that the interviewer is already providing?
I personally never work from verbal descriptions, I document the issue, and place it into a ticket system in which I work the tickets based on priority.
If you are this incapable of listening to a two or three sentence problem description, ask the interviewer to print it out for you and write "ticket #1" on it? I mean is your nitpick really "They told me the problem out loud instead of writing it down?"
This is even ignoring the fact that taking verbal problem descriptions is an incredibly useful skill as a software engineer. I mean I guess I could tell people to file another ticket and I could have the joy of e-mail tag as I have to send off clarifying questions further delaying the task.
I assure you I am not trying to be argumentative, I am trying to help people understand why I and the author of the article believe that it is a flawed hiring practice. I have raised some good points, you can reflect on them and utilize them or you can discard them as rubbish. But I can tell you this, I have a lot of success in recruiting and I have learned by committing some of the very mistakes that I now advocate against. I used to buy into the Cargo Cult hiring practices, until someone smarter than me taught me how to truly identify talented individuals.
That's not the goal of the exercise however, nor is it a desirable one. The point is to a very loose test of basic ability. You can test the basic ability of a runner by asking them to do a quick sprint; the fact that this is not an accurate simulation of a race is irrelevant.
> do you think that they will be comfortable making those small errors in front of peopl
They should be. Any programmer should be comfortable making errors and owning up to them.
> Do you think that they jump into problems and that small errors are OK to them In this situation it is. In a "real world" situation, those would be caught by the compiler or unit tests. Of course really bad mistakes in this test would count against you.
> They may blank, they may freeze
Of course, this can happen in any type of interview. The point of interviews is to try and get the best balance of weeding out unsuitable people and finding the best ones. No matter what process you use, you will occasionally discard perfectly viable candidates...however this is preferable to accepting unsuitable ones.
> just be the type of person that reflects on an issue for a while before the set out to code.
...and that's fine...I'm not sure why you think this is some sort of "gotcha" test. It's perfectly fine to take some time...nobody is timing you. Again, this is a basic test to make sure you have the simplest skills and to get a little insight into your thinking/working. If you are uncomfortable with some aspect of the test, it is fine to say "well usually I take some time to think" or "I don't like writing on whiteboards"...etc.
You seem to be fundamentally misunderstanding the nature and point of the exercise. Errors are ok and expected because it's a simple test of reasoning and logic ability...likewise jumping in is ok because it's a simple problem.
Do you as a programmer spend a lot of time considering a simple if/else statement? Of course not...you jump in and write it. Again, we are talking about a simple algorithm here...given the description it basically writes itself for anyone with a modicum or programming experience.
The point here is to get a basic assessment of raw ability. The rest of the interview is to determine how well they might fit in the company, and any other gaps can be filled in with training.
> I personally would search for the mathematical formula and a text description that I could put on my second monitor as reference
That's the point you're missing. You don't need to search for that, it's been given to you. The description of the triangle contains the algorithm for constructing it. It's also so simple that you should be able to hold it in memory for the short time necessary, but if needed you could ask for a refresher.
> But first I would search for a standard library and avoid the effort all together.
That would negate the purpose of the test. Again, this is the most basic of basic programming tasks. It tests your ability to write simple logic and flow controls...something fundamental to writing computer programs.
Even if you solve a problem using mostly library functions, you still will need a few simple loops and logic to tie them together, and that's what this is testing.
> I personally never work from verbal descriptions
The verbal here is incidental...the point is you're working form some description or set of requirements. In this instance it's so simple that a verbal description will suffice, perhaps with one or two reminders. If you really can't hold such a simple description in your mind for 20 minutes I'd say you're not suited for any sort of mental work and might need to see a neurologist.
It seems your objections are based on a flawed premise, a fundamental misunderstanding of what the test is, how simple and easy it is, and what it is supposed to be testing.
If you look at want ads for day labor and other physical tasks it will often say something like "must be able to lift 50 lbs". When you went to the job site, they might ask you to lift a bag of concrete weighing 50 pounds. This is an exceedingly simple task and anyone fit for the job will be able to do it easily. It will quickly weed out anyone who is simply unfit for the work required, anyone who is fit will be able to do it easily.
More specific to your answers I understand your position, and I truly believe that you use it in this manner, the problem I have is that for every one of you there are 20 of the former me's who get so wound up in code tricks and trivia that we create unfair and useless interview filters. As such there is a perception among developers about code test, and that perception can be summed up that I am going to be tested on what the company thinks I should know, not what I know. For some of us, this creates a horrible and humiliating interview experience. In which a developer stands in front of a whiteboard analyzing their every move. No matter how much someone reassures them, they still have everything but logic running through their mind. It only takes one interviewer with an ego to permanently affect a developer in this manner and no amount of reassurance from a person they just met will help someone believe that their every move is not being judged.
Many people with Asperger syndrome fail tech interviews horribly but a good deal of them turn out to be wonderful developers. My best friend is affected by it, and I know from experience that he would never land a job if it where not for me. In spite of that, he is always the star developer at every organization that I bring him into. Now if someone sat him down and said, show me something you built and walk me through a routine, like the article suggest, they would be fascinated by his depth of knowledge.
If you don't know the formula and they haven't volunteered it, you can simulate this search simply by asking, and any interviewer using this for a proper purpose will happily write it on the whiteboard for you and walk you through it if anything is still unclear. It's not what's being tested.
What is being tested is that you know enough about programming to understand pretty much nothing more than basic flow control, basic arithmetic and perhaps trivial IO.
A use of a question like this checks that you've at least read "programming for dummies", and roughly understand the overall concepts of programming not whether or not you're proficient enough to even fill an entry level position. It weeds out the candidates that are completely unable to program at all.
> Do you think they will be able to put there best foot forward in such an unnatural situation? They may blank, they may freeze and they may just be the type of person that reflects on an issue for a while before the set out to code.
I've had smart candidates blank on trivial problems, but there's a very noticeable difference between people who blank and people who just have no clue. With people who blank, you get them to calm down, you ask probing questions to get them talking, even if it's about the weather, and they will eventually start offering up small bits and pieces of an answer that makes sense and gets them to "unfreeze" and start solving the problem. Whereas with the type of candidates this type of interviewing should be used to get rid of, you get absolutely nothing and/or total bullshit.
> But first I would search for a standard library and avoid the effort all together.
That's something you might say as a "by the way", but it isn't what the interviewer will be looking for.
> I personally never work from verbal descriptions, I document the issue, and place it into a ticket system in which I work the tickets based on priority.
Fine, so consider the description on the whiteboard the ticket with your highest priority would be my response to that if a candidate were to raise this objection (though be honest, if they did alarm bells would already be going off in my head).
Once in a while i have to look on how to format a string and how to use map, reduce, etc..
do i have to go to school because of that? heck i dont even remember what i had for breakfast yesterday
For this sort of test, all that is necessary is that you understand the basic idea of a for-loop...the basic idea of how to format a string...remember there's a good chance you're writing in pseudo-code.
Presumably however if you are a programmer, there is at least one language where you should be able to complete this task without looking anything up. It requires only the simplest of language constructs/library functions.
The exception of course is if this is a job for say, a Python programmer...and on your resume you say you are a "Python expert". The fact that you have to look up the syntax for basic constructs in Python would suggest you were lying on your resume. That's one of the things this exercise tests for.
If you said in your resume "I have experience with other languages, but I can learn Python"...then just being able to do this exercise in pseudo-code or any language of your choice would show you have basic programming skills and should be able to pick up Python.
If you really have to remind yourself what a for-loop is then yes, maybe you do have to go back to school/hit the books and study harder.
You see that's the crux of the issue, I can be a JavaScript expert and deliver massive JavaScript applications without having to resort to the arguments array to figure out how many parameters where passed into a function. Also, if I ever had a use case for it, I would look up the API and probably mentally discard it shortly afterwards. The interviewer thought that it was a perfectly reasonable filter, but yet I am one of the top JavaScript talents in my market. It did not work for them, because they assumed that knowledge of the arguments array was basic knowledge and any developer worth their salt would know it. Yet in my time working with some great JavaScript developers not a one of us ever used it. Further it harmed them because I now believe they are as incompetent as they believe me to be, as such if someone in my peer group asked about a position with them, I would relay my story. Not to be indignant but to be truthful, as such they lost access to valuable resources in a constrained market.
That being said, one can be an expert in say Java and not know the JDBC API, maybe all of their work revolved around JPA. Many in Java would argue that JDBC is fundamental and should be known to be an expert but a new crop of developers have never know anything but ORM. But we older developers assume that JDBC is a fundamental prerequisite because it was in our time. They cant see anything but the progression from JDBC to JPA because it is the life experience that they had. As such it's flawed and irrelevant. You have to be very careful to not bring interviewer bias into an interview and test are loaded with them. Even the simplest of test.
Does it not strike you that quite a few HN'ers openly admit that they would not pass the filter? Some of them long time contributors and members, some of them very respected? To me, I would question the viability of my assumptions if HN'ers where openly saying no your filter would eliminate me. I mean we have some of the best of the best lurking here. 37 has been complaining about it for a long time, I have had my own threads about it, and everyone agrees that hiring is flawed. Maybe it's time we reevaluate our assumptions based on what guys like the crew at 37 are doing since they seem to be improving on candidate selection over the rest of us.
The point I am making is what you are trying to garner, can be identified by asking a candidate to show you something they built and walk you through the routine they are most proud of. There are going to be functions, loops, if statement etc. in that code, if they can't explain what they built, then you should be suspect. Giving them a test just introduces bias of what the interviewer thinks they should know, and does nothing to tell the interviewer what they know.
To be quite honest, this thread is having the opposite effect. I'm questioning the quality of HN contributors based on their inability to either comprehend the question or code a simple algorithm.
I passed similar coding questions in interviews before I'd ever written a line of code in my life (in a Microsoft PM internship interview). No one cared if I wrote a semicolon or not - the interview was asking if I understood basic concepts like loops and logic.
I have yet to see someone who says that they would not pass the filter. Every single example I've seen in this thread is someone making up a different filter and saying that they would not pass that. I see people saying that they took a long time to debug subtle bugs in it, that they don't know the definition of the problem, or that in general they need documentation to code, all of which is completely irrelevant to the filter as given. Subtle bugs are allowed, the definition is given to you, the solution doesn't need any documentation (and if it does, you can just make up whatever API call you expect would be present in a sane standard library)
I'd be shocked to find any good programmer who, given the definition of Pascal's triangle, couldn't produce some decent, possibly buggy, incomplete, and syntactically incorrect, code on a whiteboard to print out a row from it.
I've yet to see anyone give an argument for how a good programmer could fail this test that doesn't rest on some feature of the test that doesn't actually exist.
You are now hitting at the core of the issue, to most developers bugs, bad code, correct API's matter, you are asking them to look past something that is engrained in their work ethic, personality, and quality as a developer to measure their worth as a developer. It does not matter what the filter truly is, their is a perception on the developers part of what they are being tested for, because they attribute what makes them of value to the test. As such the correct solution is what makes them valuable. You can't tell them to give you what they perceive to be the incorrect solution and expect it to accurately reflect what they value in themselves as a developer, that the interviewers bias. The fact that the filter is being misinterpreted is irrelevant or very relevant depending on your perspective. To the interviewee, they are going to perceive the task just as many on HN have and no amount of prodding to give up their nature is going to make them perform well, conversely they are going to perform worse. I have neglected the fact that the filter is being misunderstood for the very fact that, it will be misunderstood and for that alone it sets people up to fail.
This is one of the point that I am getting at, you are right it's not just the code that is being judged, candidates know that too, so when they are standing at the whiteboard a million things about being judge may be going through their mind. It has nothing to do with being stodgy or honorable or above the effort and everything to do, with some people just cannot perform in such an environment. People hold out that it is a negative filter, that if you can't pass it, you are a definite negative and observations on a lot of peoples parts are not confirming that assumption. I side with the author of the article that there is an all inclusive way to identify good candidates that fall into this category as well as candidates that would pass the whiteboard test without resorting to the whiteboard. I am not arguing that people should not be able to code, rather I side with the author that there is a better way to identify the people that can and receive more positive hits in the process. Thus reducing the interviewing cycle and identify more talent in a constrained market.
I see where you're coming from. I think the issue is that 99% (100%?) of all HNers would likely be great hires for most companies, so they see these questions as trivial, dumb, and not representative. Just the fact that someone thinks about these types of questions and reads programming blogs probably puts them ahead of most of the programming population.
The problem is that many of us have really never seen the mythical horrible programmer. The person that really can't even start a solution to FizzBuzz. The one that leaves the interviewer wondering how the person got jobs at other companies. I have seen and interviewed them before and it's frightening and uncomfortable for everyone involved. I've had candidates break down crying when I asked them to write a simple sql select statement on the board because their resume said expert in sql. For many, it's hard to fathom these people are out there, but they do exist.
FizzBuzz style questions are really important (eg. create an array with 4 numbers: 1, 2, 3 and 4 and output the values. Oh a "Java and Python 'expert'?". Please in both languages then!). Although I agree that home tasks and trick-algorithm questions are useless.
I don't think home tasks are useless, but they have to used properly. A home task geared to 1-2 hours seems reasonable for an exercise between the first and second interview. Some may say they'll just look up the answer (it's encouraged!), and that's why during the second interview the employer should discuss their answer. If the candidate can explain why they made particular decisions on the take home piece and have an intelligent discussion about the trade offs they took then who cares if they looked up the answer. In fact, the home test is probably closer to real programming since I don't know anyone who codes without Google nearby.
So, my preferred method is to use FizzBuzz as a first cut on the first interview and then give a take home quiz that should take the candidate 1-2 hours or less if they are good.
Any programmer I want to work with will be able to walk up to a whiteboard and write a program to print a row of Pascal's triangle or a function that determines whether one string contains another. Simple programming problems like this are the tiny problems we solve on a daily basis; if you can't implement such trivial functions without bugs, how can you expect to design or implement multi-kloc systems without bugs?
Until I get a keyboard with a tappety tap tap under my fingers my brain is at all stop.
Tom Marra's don't get hired at these big web co's kind of places.
This is probably the reason why there aren't many Tom Marra's at these big web firms and why don't seem to make any thing awesome anymore, than their first initial offering.
The job these algorithm kids in these big companies seems to do is to some ensure the wheels are rolling and they don't punctured. If they do, fix them by some fancy hackery!
I still contend that you are not gaining much insight and are producing an unnecessary filter that can do you more damage than good. I was on a project once that was cursed, we had a developer have a stroke, I had to have my heart restarted due to SVT from sleep deprivation etc. etc. We literally lost everyone that knew the technology except for me and I was not in good shape, and there was this one guy Tom Marra, he was an artist, but the kid had passion he had never coded in his life, but we where in a really bad situation, once he found out that with source control he could hack the crap out of the code and then just re-pull to fix his mistakes, he set out to teach himself JavaScript, on his own initiative. In less than a month he was contributing code to source control and in 2 months you could not tell his code from a seasoned pro's. The one quality this kid had was passion. You can't identify Tom Marra's by making them solve trivia on a whiteboard.