- Brute force is often simple to understand. That directly translates into maintainable code.

- The business constraints on the input size may make the brute force an acceptable solution.

- If you need a better optimized algorithm, brute force is a great place to start. It's basically your test case generator for verifying that your more complex algorithm is right.

After they get brute force code written, the coding part is over and the rest of the interview is about discussing why it's brute force and what we could do about it. But no more coding is expected. It's a chance to be creative. If they want to talk about improving the algorithm that's fine, let's draw some diagrams. They want to throw more hardware at it? Sure, let's talk about how to scale that. Maybe they've actually seen a business problem before that was similar? Great, let's discuss how you solved it.

I still remember a recent interview I had with a team at Amazon.

That team had accomplished NOTHING (and I mean NOTHING OF SIGNIFICANCE) in past 3 years, went through re-orgs and new managers, high attrition, nothing to show for.

Here I am with successful products, published code and a track record of delivering. I have the skills to help them out.

What happens? I get rejected because I didn't remember how to use a comparator in merging k sorted arrays and write that perfect code on a whiteboard.

WTF??!! Did hiring rote memorizers really help that team in Amazon? Why do they not have something productive in 3 years despite having hired experts at tree traversals?

Well, sure, but my experience has been that the technical interview exam is far more elaborate than this. My questions were more along the lines of finding all permutations of a set (usually with some twist, like find all permutations that, when combined, match another member of the set). Or finding all matching sub trees in an binary tree, or something depending on merge sort or quick sort. That sort of thing. Nothing impossible, but considerably more involved than simple recursion. And the teams that interviewed me clearly did expect to see largely working code written on a whiteboard.

Overall, it's elaborate enough that people do need to study substantially for their interview exams, and good developers may simply decide they don't want to waste the time on a job that might not even come through anyway. I once learned to do intricate integration by parts in calculus, but I wouldn't be especially interested in training up to do it for a data science interview. If it comes up for reals, I'll deal with it.

This is something the industry has inflicted on itself, which is why I feel a lot of irritation when I hear the same companies talk about a severe shortage of talented software developers.

Interviews are a mutual process, where the hiring team must do their best to persuade me to join. (The same way I do my best to persuade them to accept me.) Some of them utterly fail, even if they make an offer.

Indeed. Now the default seems to be companies sending out a Hacker Ranks test consisting of 5 separate problems, a countdown timer and zero human interaction.

I can't help but think that recruiters are playing a part in pushing for this online Hacker Rank test bullshit.

It feels to me that as the quality of people working as recruiters goes further into the toilet the reliance on the Hacker Rank puzzle test seems to increase.

This depends on the problems being reasonable, though.

Sure, for various reasons, most programmers know the factorial, but I'd consider remembering the definition of some mathematical operation only common in certain domains (statistics and finance?), that most people only learn in high school and never use again, to be rote memorization.

Or you can use a search engine.

I actually don't have a problem with the data structures interview exam inherently, my problem is with how it's administered.

It doesn't bother actuaries have to take exams to demonstrate an understanding of linear algebra, vector calculus, stats, numerical analysis, and so forth. Nor does it bother me that they have take these exams even if they took the material from a reputable college with a good grade. I think it's great that the actuarial field will allow people from multiple educational paths to take the test, giving them a choice about how they learned the material.

But here's the thing - software developers have to take these "interview" exams over and over, at the whiteboard, every time we look for new job. And unlike the actuarial exams, we take these tests under conditions of tremendous secrecy. We don't know if our examiners are qualified, or if their questions are vetted, or if the exam is consistently applied and graded. We don't have a clear and defined study path, aside from books like "cracking the coding interview". And we get no feedback, hell, at times we don't know if our performance was evaluated at all.

I've considered this for a while, and I've come to understand that exam-based professions and institutions, such as the medical boards, the bar, universities, have evolved a set of considerations, kind of a student bill of rights. Students must submit to the exams, but they have the right to a fair and consistently administered exam, evaluated by respected people in the profession or field, that the exam questions should not be capricious or utterly unexpected, that there is a preparation path, they get feedback on why they did not pass. What we have in tech is the stress and burden for examinees, in many ways amplified (whiteboard coding, on the spot, is among the more stressful ways to take an exam). But the student bill of rights is largely non-existent, and I think this is the deep and severe problem with tech interviews. Until this changes, to me, nothing can change meaningfully. And I don't just mean in interviews, I think this sort of thing is what is deeply rotten about tech, at the core of many of the problems (in a field where there's a lot of suspicion about bias, are secret tests under capricious conditions a good idea?)

And we're expected to come back for more of this, over and over, and to just take it.

Oh, by the way, there's a shortage of software engineers, best solved by putting tech companies in charge of the immigration system so they can decide who is and isn't allowed to come to the US based on how well they do on these interview tests/how much they're willing to put up with these interview tests.

One way in which we're approaching the stress of interviews is to strip away the bad outcomes and allow people to approach interviews as "there is nothing to lose". Another way is experimenting with different types of interviews such as group interviews or project based interviews. Sometimes the issue with those experiments though is that they're even harder to standardize.

I think that in addition to what you said, we need to get to a measurable level of improvement. We try to measure the repeatability of our sessions in two ways: 1) when the same session is evaluated by two different interviewers, the ratings across dimensions should be equivalent and 2) (this one is harder to experiment control) when the same person does two interviews, they should be evaluated equivalently on the intersection of sets of things that were evaluated. By optimizing for these two ways of repeatability, we are hopefully going to move toward much more objective driven evaluations that capture the engineer's understanding of different areas and not just the spur of the moment.

Based on what I've seen from your site, this does look like an improvement.

An interviewee can prepare properly, and take the interview once prepared, rather than doing this at arbitrary times that may be very busy, simply because an interview came up.

Because the interview results can be used in multiple places, there's no need to do this repeatedly for companies that have confidence in the exam (I hope!).

There are reassurances that the interviews are conducted by experienced engineers. I'd be interested in hearing more about this.

There is actual feedback, which is critical. I interviewed at google, and my understanding is that there are actual, numerical scores in a database for my performance in the various interview exams, but I'm not allowed to know what they are. To me, that's a huge problem, and I'm glad to see from your example that you provide feedback.

As far as the experience of the interviewers goes, we're pretty careful about it in our on-boarding process. Even more than the experience we look for open-minded engineers who are willing to question their evaluation and constantly work on improving the overall process.

Re: feedback at Google. It's hard for companies to give you access to feedback because of potential legal repercussions and also because it is very hard for them to deal with back and forth that would inevitably occur.

That's another big point that we emphasized early on that seemed like a crazy proposition - fully transparent interview process. That is, anything that a company would see, you see beforehand.

Although I agree with this point, it only explains why a bad practice occurs, not why it should happen. If anything, it indicates why google might not be a good institution to be conducting exams. What you're saying here (accurately) is that because Google could be sued for how it evaluates and uses its exams, they just keep the results secret.

To drive the point home - there is currently a class action forming against google for age discrimination. However, the process is opaque enough that people probably don't really know if there has been any age discrimination. One possible data point that might be of interest would be to see the exam results, or other information about how the exams were graded (I believe there are even images of the code I wrote stored in this database for google's record keeping - but not mine. Again, I'm not allowed to see them). So, one way to discriminate and get away with it would be to simply deny anyone access to the data used to evaluate them. That way, how would they ever know?

I understand this is true of all interviews, not just tech. But tech really is unusual (perhaps unique) in that a huge part of our "interviews" truly are exams. They're not "tell me about your experience". They're "write code to find all sub matrices in an NxN matrix with a positive determinant", often at the whiteboard, in 45 minutes.

If that sounds extreme or excessively suspicious, I'd like to point out that we're talking about a company that was clearly, glaringly guilty of an industry wide collusion to suppress wages and maintain no-hire lists. I'm hoping you've read those articles and seen those emails. It's reasonable for people, at this point, to be looking for some kind of outside regulation and transparency.

1. Don't ask questions that are hard to explain over phone. For instance I was asked if I've used Lambdas in C# and I said, "Yes, extensively.", next question was. "Explain to me what a Lambda is.". I gave the textbook definition for it, and he was expecting more, so I simply said "It's hard to explain lambdas, I can show you some source code I've written and that way I'll be able to explain it better.". I thought that was it and that I would be 'screened-out', but I was moved to the next round of on-site interviews!

2. Do ask questions about how the candidate has used the technologies that the job requires, and ask them to explain how they designed and implemented their most recent solution, and where they used what technology, and how. After initial answer, drill deep into areas that are important for the job posted, and from the answers, you can get a fairly good assessment of whether the candidate should be moved to the next stage of the process.

3. After passing the tech phone screen, I've received and completed 3 take-home coding exercises (1 each from 3 different companies), and in 1 of those companies, I was subject to white (water-) boarding exercises on site in addition to the take home test. The white boarding seldom goes well because of the environment and nature of the whole thing.

I found the take home exercises to be better than on-site white-boarding because the former is closer to how a future employee would work, than the latter.

Also, how a candidate does on algorithm and data structure questions is a false positive (or negative) because these can be practiced, rehearsed, perfected and hence gamed. Coding exercises, or specific question on the candidate's most recent project, on the other hand, cannot be 'gamed' and even if it was, you could tell from listening to the response, or from reading the code...

Hope this helps!

(In a previous role, I gave applicants the same -- very trivial -- question that they had already coded the answer for in a do-at-home screening. I wasn't secretive about this, either, I told them that this is the exact same question they had already answered. A significant percentage of them were unable to figure out the same answer they had provided a few days earlier.)

I never understood this. In a programming / software development job more than any other job, aren't you very likely to fail at delivering on the job, if you cheat on the interview and tests?

If you're pretty happy with your routine the next step is review. Look at candidates who passed a phone screen but were later turned down at a followup stage. Is there anything you could change about the phone screen so that the later time and money could be saved? (If you have no one who passed the screen but failed the later stage, sounds like you could just eliminate the later stage altogether.) Another interesting thing to review is candidates who didn't work out after hiring, or didn't work out as much as hoped. Again, anything at the phone screen that could have caught this? Or for those ones you can look at other stages too. One example for me is an intern that didn't work out so well, we only do a couple phone screens for them though, but still the biggest skill that was missing that I realized I could try testing for was the ability to dive into unfamiliar code they didn't write and make a few small changes at various places to fulfill some business goal. Or in other words, filter out people who can only program if they can keep the whole program in their head (which usually only works for larger things if you wrote everything yourself).

I think only google needs to decide this. Everyone will just follow what google does.

"If you have kids, you're too old to code and should move into management." This is basically what the industry is saying.