Ah yes, the bane of my existence, again: "senior" == "ability to build large distributed systems".

I mean, hell, that's a great skill. I wish I had it. Then again, I've seen people forced to build "large distributed systems" to solve things I can solve on one thread on one CPU because they don't have my skills. It's almost as if there is more than one skill that qualifies you as "senior" or something!

What is funny is I had spent more than half of my career working in the SEO space building "efficient," "high-scale," "single server" web applications, because people don't want to have to hire teams of people to manage all the different "-ops" that go along with the modern, micro-service, distributed architecture that is unnecessary 90% of the time.

All of those quotes were because they are all relative terms. Like high-scale meaning 10,000s of concurrent users; and single server for the web app (separate database server, or utilities server for cron jobs, offline processing, etc); and efficient meaning nothing really.

I don't think every system needs to be large or distributed. 90% of engineers aren't going to need to go beyond the basics of load balancing.

What we do is just that, so we're trying to see if a potential hire will be able to work with what we have.

Non-senior SE candidates don't have to do this exercise.

I have my doubts about your conclusions given your statements. It seems you have 3 steps in your interview process: code assessment, architecture assessment, and CTO interview. And the candidate who does well on all three gets the job? So, does your funnel filter to exactly one candidate every time? And if it does not, then what criterion are used to funnel all candidates who finished this process successfully? Perhaps, you will argue it comes down to whomever did better on one of the exercises. But that is never a guarantee and would seem a trivial differentiation if multiple candidates had viable and correct solutions.

So, genuine question then: What are the final and most important criterion that would be used to differentiate between two completely successful candidates being equal in their performance in your pipeline?

I think the answer to that will give more credit to the comment you are trying to negate.

In my experience, if there are multiple qualified candidates it usually comes down to things like who lives closest (so we don't have to pay to move them here)? Who was the most friendly/likeable? Who might have a skill that we might need in the future? Etc.

Asking candidates where they live is illegal. You can not discriminate based on exact location. (It’s okay to ask if they need relocation of course, but you can’t decide based on who is “the closest”)

I don't need to ask them. I've never ever seen a resume (and I've seen thousands of resumes) that doesn't have the candidates home address at the very top.

> to things like who lives closest

Yikes! You may want to review laws about hiring... immediately.

Have you ever seen a resume that doesn't have the candidates address at the top of the page? I haven't. We typically receive hundreds of applications for any one job. We phone interview maybe 20 of them and in person interview maybe 10 of them. Those who we don't hire are never given a reason why we chose someone else. They don't know if it is because they don't have the requisite skills or because they live 2000 miles away and we would have to pay to move them to our state.

This isn't to say that we never hire anyone from out of state. We have done so on many occasions. But if there are 2 equally qualified candidates, we are more likely to choose the one that already lives close by and can start immediately.

These responses cast serious doubt on some of your earlier comments.

In what way?

My experience as a hiring manager disagrees with yours. Large system design interviews are par for the course - everyone knows how to do them and everyone has lots of experience. They don’t tell you really anything about a candidate, to the point that we should stop using them.

In my experience, this has been rather selective.

Some people have very little idea about load-balancing, DNS, database scaling (replication, sharding, etc), fault tolerance, graceful degradation, queues, throttling, caching, you name it.

If candidates take a course like this, does that mean they're self-starting go-getters trying to expand their knowledge to become good engineers, or people trying to game the section on the interview- or both?

https://www.educative.io/courses/grokking-the-system-design-...

I thought about this myself as I went through this exact course. For context I had been exposed to and even implemented some of the concepts (like db sharding) at work but much of it was still new to me.

An ideal interview should reflect the actual work you'll do during the job. And I think system design interviews accomplish that job pretty well (certainly much better than coding interviews). Will you ever have to design and build a system like you during the interview? Likely not - a startup, while does require building a lot things for scratch, rarely requires the scalability and intricacy outlined in these interviews. While large companies will have the components broken up by team, so you'll often be silo'd into just one part.

That said, knowing how your systems work at a high-level is much more useful than knowing something like shortest-path algorithms. Even if you are silo'd, understanding upstream/downstream interactions can be crucial.

If they actually understand the stuff they learn this way, I'm fine with that! It's strictly better than just doing nothing.

These topics are covered extensively in cookie-cutter interview prep materials like Hacking the Coding Interview. Every candidate is treating these things the same as leetcode trivia. There are whole YouTube channels devoted to how to rote memorize the path to answer open-ended system design questions like designing a fault tolerant Twitter feed, asynchronous video streaming, etc. The rote memorization even extends many layers deep into “nuanced” follow up questions that are supposed to distinguish mere rote memorization from real experience.

This is just every candidate, certainly at the senior level. It’s impossible to use these questions to discern anything. Even asking them to give these details about past real systems they designed you’re just getting the same 5-layer-deep inception rote memorization of everything they could be asked and every follow up contingency.