In my experience (also interviewing hundreds of people) this is totally an urban (internet) legend. Not once has someone been able to talk intelligently and at length about a project they built and the underlying technology choices and then been unable to deliver. Once you gain experience building things yourself, it's pretty easy to suss out whether someone is any good just by talking to them. No coding tests needed.

Agreed. The best way to interview is to simply have a joint discussion about their past projects (from resume). There is no value in whiteboard puzzles. It is easy to gauge who is competent and who didn't do or didn't understand the work they claim on the resume.

In over 25 years of interviewing candidates, this approach has never failed me. I've never asked for whiteboard coding. Whiteboarding architectural concepts, sure.

The only caveat is the interviewer needs to have substantial experience in the same field to spot the wheat from the chaff in a casual conversation. If the interviewer is very junior, I can see how this might not work. Solution: don't have your most junior people run the interviews.

I use the same approach. Also has never failed me. If you can code system X, then you know how to use an if statement.

I've interviewed hundreds of candidates over more than 20 years. I've lost count of the number of times that a candidate has talked-the-talk so well that I'm thinking "This guy/girl is amazing. They're a shoe-in. Better do a little coding just to make sure"... and then they literally couldn't even get close to solving something like fizz-buzz.

I agree asking highly specialized, deep CS-algorithm type problems is usually a terrible indicator. I typically start with a very rudimentary question that could go off in various directions and to various depths as time/progress allows. I let candidates choose the programming language, and to use either whiteboard, pen & paper, or laptop with choice of IDE.

I've seen candidates who claim to have years of "senior" experience struggle to get the basic syntax of their "language of choice" right - to the point where a competent hobbyist who spent a weekend doing a couple tutorials would wipe the floor with them.

There's just something about coding that makes it impossible to truly evaluate a candidate (for a developer role) unless you see them actually code.

I've not had this experience. How are you screening your pipeline? I have a hard time believing that someone who can tell you why they designed a particular REST api the way they did, describe the schema and talk about what performance issues they encountered and how they dealt with them and can intelligently discuss the language they're using but can't code fizzbuzz.

My guess is that either your interview is too much fluff, and not enough technical detail, or you're intimidating the candidates and they get lost.

It makes zero sense that someone can have a deep technical discussion but they're not able to write fizz buzz (which by the way is a crap exercise). Sure, if one is asking general questions about agile, TDD, SOLID and other fluff, anyone can answer those.

At one of my previous gigs, we managed to hire someone who couldn't write more than basic control structures. I have absolutely no clue how it happened as I wasn't involved with the interviewing process at the time. The people who were involved blamed the recruiting firm for not doing adequate technical screening, but that doesn't explain how this person passed the on-site interview they performed... At any rate, we let them go after 3 days.

I have no way to prove to veracity of my story, so feel free to add it to the "urban myth" pile. But it was extremely bizarre to experience.

I agree with the urban legend guess.

Yeah it happens, but not as often as you think.

Acing an algorithm type interview has little to do with answering questions like can you translate business problems into working systems? Do you know whether you should be writing this code at all or use a third party solution? Do you know how not to gold plate a solution and ship software? Can you solve a performance issue with a process that may not involve code but involve another part of the stack?

We have ~4 hours to judge a candidate. Like it or not, we are going to have to choose :)

The complex questions, if I can get answers are awesome. They are also the ones that are hard to objectively describe. Based on the number of blogs read, candidate charisma, my own work pressures - there is a big chance a senior enough person can wave their hands through those. Hence the tilt towards the surety of coding.

Also TBF, in my 1000s of interviews, we have always had a couple of rounds (Bar raisers / System Design / Architecture ... whatever) which try to get the fuzzy input. We get a sense, but that's not in depth as you might imagine. OTOH we have a clear cut answer in the problem solving world that gets us a bit more closer to the hire/nohire decision.

I’m on the opposite end of the spectrum. I work for small companies mostly and I’ve conducted interviews for small companies. We have a limited budget for a limited number of positions. One developer can be working on a project that can individually move the needle. We need to hire “engineers” in the truest sense of the word not “coders”.

For instance, currently my title is “senior software engineer”. But I am the “Directly Responsible Individual” for a project that one of our clients is paying for. It will be added to our core product.

This means that I’m expected to interact with the client and BA to come up with the SOW, work with the product manager/UX person to design the interface, do some of the web work, write the backend using one of our approved languages (C#, Javascript, Python) , design the database schema, do all of the CI/CD setup and to know the AWS fiddly bits and know which services are appropriate for the asynchronous processing requirements.

I’m no special snowflake, this is life in small companies since my first job 20 years ago.

I don’t interview candidates for algorithms. I interview them to know whether they can hit the ground running on as much of our (industry standard) technology stack as possible, will they make good architectural decisions, can they ship a feature across the finish line, how fast they pick up new frameworks and will they embarrass us with customers and senior management.

LeetCode ability does us no good. I want to know whether you can translate real world business requirements into a working system - including knowing which questions to ask when customers/product managers come to you with “XY Problems”.

I don't know your scale for small, but my current org is tiny compared to AWS (my previous).

So yes - we have to do all those things. I am directly responsible for ALL of data analytics infrastructure in snapchat. Pretty much most senior engineers have a similar scope.

We might be talking over one another though: I am not saying any of that is useless: I am saying they're hard to measure. We still do it though. We also do the coding round as something that gives a more definitive datapoint.

My definition of small is less than 50 people in total including salespeople, managers, implementations managers, and a few QA people.

It’s really not that hard to ask a few technical questions based on what they purport to know, the soft skill behavioral questions, some design questions, and ask them about previous projects. The hands on part is the part where they fix the code to make the unit tests pass.

I wasn’t asked a single technical coding question for my current job. I was given real world questions about issues they are having with process and how I would go about (re)architecting some processes.

Now that I think about, my job before that when I was interviewing for the dev lead position I was only asked about process, system design, and a lot of high level theory - both jobs were hands on coding positions. Both of the managers were highly technical with current programming experience.

But, I’m not completely disagreeing with you, if you are solving hard problems(tm) at scale you need people who understand algorithms and you may not be able to use off the shelf solutions - I would say Snapchat qualifies for “at scale”.

But, for you typical yet another software as a service CRUD app or your typical dark matter corporate developer - and those positions are a lot more numerous than HN seems to realize - the complications aren’t in the complexity of the code, it’s in translating the business needs to code and shipping software that the company needs.

As strange as it sounds, the “coding” side of software engineering has gotten a lot simpler 20 years into my career than it was when I started (C, C++, and a little x86 assembly). The “engineering” side has gotten a lot more complex and requires me to know a lot more technology.

I don’t think it’s because I am better “coder” now either. When I got my first job, I had already been a hobbyist for a decade doing assembly.

Ummm... weren't you claiming in a comment the other day that you use FizzBuzz to interview candidates?

No, I was defending the use of FizzBuzz as a “Bozo filter”. But FizzBuzz is so commonplace that it is meaningless.

I have been able to weed out candidates by asking them to do an isOdd() function as part of a pair programming exercise. It was still the Bozo filter before I gave them harder problems. Similar in complexity. But even that, I’ve done with an IDE with a skeletal class and failing corresponding unit tests.

In my experience, I've seen close to 50% of the people that can talk their way through an interview and put up a good game wind up being absolutely useless at actually writing code once you hire them and set them to work. There is a huge disconnect between being able to understand the big-picture hand-wavey ideas of how software fits together and being able to turn that into nuts-and-bolts working code.

Something as simple as having somebody write a little web application that takes a request, calls an external service, parses the response, and returns a transformed response, filters out the bozos wonderfully.

An alternative hypothesis I have is that some organizations inadvertently skew their pool somehow, and they really do get a lot of these, but other places don't.

The problem with non-coding interview's is its very dependent on the interviewer to determine their capacity, and _many_ people are bad at that. I think if you are good at it, its pretty easy to tell (like < 5 minutes) if someone can do the job. But I've seen very poor candidates get past this screen many times, even when screened by people I otherwise trust. So I know its not a common skill.