The actionable thing is the graph about two thirds of the way down - passthrough rate vs performance.

That's the whole point of the article. Two possible conclusions:

One is that companies are cargo-culting hiring, and have absolutely no idea what they're doing. Clearly whatever the process is supposed to do, it's not hiring the "best people" by any realistic metric.

The other is that the process is working as intended, but the actual goals are not stated. This might be true if the aim is to hiring difficult and stressful for candidates to discourage job hopping. (Other interpretations are also possible. [1])

If a company wants the "best people" it needs to follow up hiring with performance tracking, and identify and reward the people and hiring practices which increase performance. [2]

If companies want hiring to continue as a social/political game which stresses and discourages developers, the system is working for them, and they don't need to change it.

Startups need to decide what kind of company they want to be. [3]

[1] IMO there may be a strong element of corporate narcissism in the process. It's not about hiring good people, it's about allowing the CEO and senior management to feel that their company is better than those other companies which hire these people. In reality "these people" may be a bit more than averagely competent, but with a few standouts most won't be insanely great or A players or 10X or whatever the goal is supposed to be.

[2] Which assumes it's possible to measure performance, which is a whole other issue.

[3] I'd bet almost anything that in reality there's a lot of "Oh, you were working with X at Y? Awesome!" affecting hiring choices in SV.

> [3] I'd bet almost anything that in reality there's a lot of "Oh, you were working with X at Y? Awesome!" affecting hiring choices in SV.

And outside of SV, seen this in the Midwest at both tech and non-tech focused companies.

> If a company wants the "best people" it needs to follow up hiring with performance tracking, and identify and reward the people and hiring practices which increase performance. [2]

And that's glossing over a ton of massive difficulty, which is one of the biggest reasons things move slowly and are more based on opinion than data.

It seems like a lot of these discussions end up being something along the lines of "the standards are bad because they exclude me, therefore there shouldn't be any standards at all" (just with a lot more words).

The standards (e.g. algorithmic questions) are bad because

A) they're only tangentially related to the work, meaning they include bad hires and exclude good hires

B) they put a huge workload on employees who must practice tens or hundreds of hours of underwater basket weaving, time which would be better spent doing almost anything else

C) they systemically exclude anyone who can't play the dumb game due to a lack of time, including but not limited to older people and poorer people

> A) they're only tangentially related to the work, meaning they include bad hires and exclude good hires

And that's the reason many companies have to set the standards so high in the first place: excluding false positives (bad hires) is so important in practice that this distorts the whole hiring process. Fine as far as that goes of course, but it does mean that people who practice their leetcode questions are basically stuck with very expensive (per B and C) lottery tickets.

I think everyone knows at this point that algorithmic puzzles are not desirable; the real question is whether one can do better, even at FAANG scale.

Puzzles no, but it really matters how candidates are evaluated on coding.

For coding, if a coding question boils down to implementing KMP string matching or something that's only realistically solvable by memorizing the entire algorithm and writing it out on a whiteboard, then it's not any different than any of the puzzle type questions like "why are manholes round?"

The proper way to evaluate coding questions, and what is mentioned in the article, is that one is evaluating general intelligence in the context of software programming. Is the candidate coming up with a reasonable solution? Are they asking questions to understand the problem better? If they get stumped by something, do they stay stuck on it, ask for help, or think through it? Can they find the issues with their code and improve on it?

I haven't heard of a better way to interview engineers, but if there is I'm definitely all ears.

I agree the standards are bad, for senior devs in particular. In addition to your points:

Technical understanding of systems in general (networking, CAP, testing, writing maintainable code) are all overlooked in the interview process, though these are what actually make a developer productive.

The older you get the more specialized and optimized you are to learning what matters, which is why most of the older guys suck at these types of interviews.

Imagine interviewing Brendan Gregg by giving him some leetcode problems!

That's a very good summary. Discerning what truly matters is really the mark of experience. And you're right, algorithm memorization doesn't fit in the top-50 of what matters for a software engineer.

I went to CMU, enjoyed algorithms classes a lot. In the nearly 30-year career since, I've used that knowledge... never. Not once.

It's not about the interview bar being too high. It should be high, mine is. But interviews based on algorithm memorization have the bar in the wrong place, measuring something that has no real-world relevance.

I think of "the bar" as being basically our industry's equivalent to a law degree: it's just a way to show that you have some foundational knowledge in the subject, even if it bears little resemblance to day-to-day work. All of the flaws you enumerated could also apply to law school, but at least "the bar" is reasonably accessible and doesn't cost hundreds of thousands of dollars.

The annoying thing is that with the bar exam, there is a clearly defined test with criteria. Whereas what you have in the software industry is a vaguely-defined set of expectations that vary widely. There's no standard.

Definitely agree with you there. I wonder how far we are from some kind of standardization. Lots of startups are already in this space, but if a few FAANG companies adopted a common standard it wouldn't be long before a large share of the industry followed suit.

ctci and leetcode seem to be the unofficial standard tbh

Yes well except you forgot that lawyers pass the bar once and don’t have to redo randomized exams over and over in under 45 minutes. My lawyer friends are making 160$/h and their interview efforts were an hour discussion with a few lunch meetings.

Not only the law degree is a standard exam, but you only have to pass it once not ten times in your career.

Most of the criticism probably should be articulated again to probe whether these standards are realistically something one can work towards.

If we’re out of the running no matter how much you practice, no matter how much job experience, and if there really is this mysterious value G, with the right combination of Ivy League CS program, then I can see how these discussions mirror the notion of ‘the bar’ - that is, a forgone conclusion.

You could start a company and hire people on other standards, and I suspect it would work, as long as the "other standards" are painful in the same way as the current ones. For example, you could require candidates to learn ten pages of Chinese poetry by heart, or anything else that tests their ability to keep their mind on-task for long stretches of time. In my experience, people with that ability can become good programmers if they're inclined to.

What won't work though is hiring based on only past experience. The bullshitters will eat you alive. You really do need some kind of "underwater basket weaving" as a test.

>And that helps us understand why Recursive Cactus spends so much time practicing. He’s training himself partially because his current company isn’t developing his skills.

I develop software full time and have some 10 years of experience depending on how you count it, and I'm absolutely terrible at leetcode style problems and find whiteboard questions difficult for non programming reasons.

I know it wasn't the point of the article, but there's a deeper problem here that most FAANG programming questions aren't actually testing work related code skills.

At my company we give two very basic tests, takehome. Both can be completed in 15 minutes if you're rushing. Far more useful of a tool than any of leetcode style questions.

I remember one time I was hiring for a JavaScript developer. The most important technical competency was a brief working knowledge of the DOM. The actual position required modifications to pages for A/B tests and jQuery was either too slow or broke in production when other developers used it for their test code.

Of everybody that passed through HR to me I sent out a notice that there would be a minor code assessment and they could not use jQuery. 6 or 7 people instantly dropped out. There were 22 people left that I actually talked to which only three could pass the code assessment.

The assessment was never designed to be challenging. Candidates were using their home computers had access to any reference material. There was no hurry. They could look things up. I also told them if they got stuck just ask me and I would point them in the right direction without any penalty, but only 2 candidates tried that.

The assessment was an hour long phone call. At the start of the call I would send out a static HTML page for the candidates to open in their browser and for them to write an answer in whatever tool they wanted but it had to execute in the console of a browser on that page. Tasks were things like take this item out of the page and reinsert at some other location or change the color of a particular paragraph to red. Stupid simple stuff that you would expect any UI developer to do easily. After all these were beginner skills and this was a senior level position.

I did ask to see the code they wrote only to make sure they completed the tasks. I would drop their code in my browser console and run. I never looked at code style or sloppiness. It’s a job interview where people are rushed and nervous. Their code is allowed to be sloppy.

Only 3 candidates were able to pass this code filter. The people that did pass either did well enough to pass or extremely excellent. The people that failed, maybe 19 of 22 people, all failed epically.

That experience really shocked me. I did everything I could think of to make this relaxing and take the edge off because there is always pressure during a job interview. No matter how much of a lifeline I threw out almost nobody would bite. They would sit there silent on the call or they would attempt to stall like that would somehow make code magically appear. Sometimes I would remind candidates they could ask me questions because I cannot evaluate a blank answer. Other times I would try nudge them to get started or guide them into progress, which only seemed to hurt more than help.

Very bizarre experience.

> No matter how much of a lifeline I threw out almost nobody would bite. They would sit there silent on the call or they would attempt to stall like that would somehow make code magically appear. Sometimes I would remind candidates they could ask me questions because I cannot evaluate a blank answer. Other times I would try nudge them to get started or guide them into progress, which only seemed to hurt more than help.

Because they felt that it was a false help. If they take it they're disqualifying themselves because they didn't know something.

I suppose they could have assumed I lied about the assistance, but surely they could have reasoned that would be less disqualifying than delivering nothing.

You need to have empathy for the person on the other side of the table.

Your relationship when questioning is very conditional. Answer question right == get the job.

I've had very similar experiences with Delphi, Jade and embedded C developer interviews. It makes me wonder why other companies don't actually care whether I can write new code, only whether I can replay algorithms.

When interviewing I build a set of simple examples like yours and run people through them, normally in person. How they talk to me is also important.

Albeit here in Australia FAANG interviews are mostly regarded as yet another US foible like driving on the wrong side of the road. I've had exactly one attempt in ~10 job changes but they failed (I walked out as soon as they made it obvious that they were serious... but were not offering FAANG money).

I've written a fair amount of vanilla JS over the past 20 years and know what you're talking about.

But in the circle of people that I've talked to, everybody else uses frameworks, and I doubt they could make surgical changes in the browser console.

So you're excluding most of today's working front-end developers, and appealing to "edge cases." I guess you could pre-screen candidates by asking, "Are you familiar with vanilla JS?"

http://vanilla-js.com/

Note: the author of that page should add React.

It seems insane to me that most working front end developers don't know how to manipulate a DOM with javascript.

Not saying this is or isn't true, but it seems insane.

Doesn't something seeming insane automatically make you question and check your own assumptions and information first? Isn't the most likely explanation for your premise either that it's not true, or that your expecations don't match reality?

Is it true that most front-end devs don't know how to manipulate a DOM with JS? I don't know, but it's not exactly true; Jquery is still javascript. Do you do front end dev professionally, and use strictly the pure javascript DOM API with no frameworks? There's a reason most people who do front end dev for a living don't do that: because it's time-consuming and there are easier options. Also because there better options than fiddling with the DOM at all, like React.

Is it insane that most programmers wouldn't be able to code something up in X86 assembly during an interview? (Not to mention that most wouldn't be interested.) To me that seems like a similarly unrealistic expectation.

I'd assume that a developer should at least have a mental model of what DOM is and what it consists of, and that it can be queried and transformed by calling functions on JS objects that represent DOM nodes.

If the dev doesn't remember the names of actual functions (which is no fault, since that comes only with using them regularly), this would at least allow them to figure out what to search for on the Internet and how to use the results.

Is that a far fetched assumption? It's not really comparable to wanting a webdev to write some assembly code.

This isn’t about having a mental model of the DOM, it’s about whether you know a specific API.

Well, the original problem seemed to be that interviewees were not able to even find the API. If they had a mental model od DOM, this shouldn't be a problem.

They'd knew that there are nodes, can be queried from the document via JS and moved in relation to other nodes, so they could just google: "move dom node", or "query dom node" and put the pieces together via JS.

Look, yes this very well might be a case of lots of inexperienced people shooting ambitiously high (you don’t win unless you try, right?). And it might be true that most people interviewing for front end jobs don’t know squat. The story above just doesn’t actually demonstrate that.

My point here is that there are other explanations, we can’t jump to the conclusion that the cause is because most people lack a mental model of the DOM, that’s overlooking many possible reasons. Nothing about the state of developers is proven by this anecdote.

The interview specifically asked about an API that most people simply do not use. It’s not a surprise that a bunch of candidates backed out when they learned this, and it’s not a surprise that it was accidentally more challenging than the interviewer thought for those who stayed.

It doesn’t really matter that Google was allowed, it’s an assumption to think that it’s easy to come up with keywords and search terms for a API you haven’t used, during a job interview. I’ve interviewed a lot of people and watched a lot of people freeze during interviews, people that I know would figure it out quickly if I wasn’t interviewing them and they were relaxed. A job interview is a test, and the test administrator isn’t in a position to claim that it should have been relaxing or easy for the test-takers.

I generally don't do front end development professionally.

I do know how to manipulate the DOM without frameworks. Learned it over a weekend.

If I were interviewing for front end, I wouldn't recall the syntax immediately, but I would know what to google or be able to have enough of a conversation with the interviewer to show them I have the basic understanding of what is happening.

I do backend. A similar situation might be something like using curl to make individual http requests.

Yeah, we don't do that in real life and I forget some of the parameters and flags available, but if I acted like I had no idea how to use curl, I hope an interviewer would be very disappointed with that.

The issue here is precisely is about recalling the syntax of a low-level approach, not whether you understand it conceptually. It’s the difference between being asked to make a request using curl, and constructing an HTTP request buffer manually without the help of a wrapper to tool to do it for you. It’d be the same as going to an interview, and having the interviewer ask you to complete an HTTP request on the spot using telnet and a keyboard. Sure, you can look it up, but when you do that you’re showing the interviewer you don’t already know.

Many devs know very well how to manipulate the DOM using JQuery, or construct the page they want using React. Most people do not use the low-level pure JS API. Conceptually I totally know how to make things happen in the DOM. Using the JavaScript built-in API, not really. Yeah it’s easy to learn in a weekend, that doesn’t change the fact that the API is tedious, hardly used in practice by most people, and rarely if ever necessary to use for performance reasons.

You say "The issue here is precisely is about recalling the syntax of a low-level approach, not whether you understand it conceptually" and in a scenario where an interviewer blindsided a candidate with such expectations I would agree, but that's not how we got to this topic.

The parent says, "Candidates were using their home computers had access to any reference material. There was no hurry. They could look things up," which is the opposite of that.

It even says they let candidates know ahead of time that's what the interview was.

I mean come on, people are on here and Reddit all the time acting like it's reasonable to grind leetcode for months to get a job, but a Saturday afternoon of studying the basics for an open book quiz on Monday is too much?

I should have said it’s about “using” the low-level syntax, and not “recalling”. You’re right that the story superficially claimed the interview was open-book and didn’t require memorizing the low level API, but it did say specifically that JQuery was not allowed. It was definitely not about conceptually understanding DOM manipulation.

As an experienced front-end developer, I can see from the story I might have personally bowed out of the interview too, not because I can’t use JavaScript to manipulate the DOM, but because I don’t want to. If the job involves doing that, I might think twice about that job.

You might be over-estimating how easy an open-book quiz is. I don’t think I’d pass an interview that asked me to write only x86 assembly, or write HTTP requests manually, or use pure JS to do what I know how to do in JQuery or React, even if it’s an open book quiz.

The big companies don't care if some great candidates can't do these kinds of problems, there are many others who can. They care about not hiring bad candidates, period.

Anyway, I've done both types and for me the false positive rate for take-home assignments is far greater. I've seen people ace home assignments and later turn out to be mediocre engineers. Maybe they were cheating. Maybe not.

But I've never seen anyone really ace an on-site coding question and not being great (I'm sure there are some, this is just my own experience).

How is your test proven to be better than the others ? Can't the FAANGs be more selective ?

We can't reasonably priced which is superior for selection, but in our opinion the process is much saner and less resource intensive on our part.

I wonder if the modern interview process at large orgs is built on a false sense of security. They are possibly overly selecting and perhaps successfully choosing above average candidates but missing exceptional cases who fail to conform to their rigid criteria.

In the numbers game they are okay with false negatives.

Those companies probably also value attention to detail and would realise that our hero is called Recursive Cactus, not Calculus ;)

> There wasn’t really any actionable thing in this article that I found, or applicable advice. The end.

This could honestly be a single-line summary for every blog post I've read on interviewing.io.

Perhaps there wasn't any actionable advice, but it's a sad indictment of the current state of tech hiring. Based on the article you can assume that Recursive Cactus is already in Big-Tech ("well known tech company") so "(a lot) higher pay" is a moot point and quite frankly it's unlikely that the daily tasks involved in the next job will bear a strong resemblance to the kind of synthetic tasks or problems you are asked about in an interview setting.

The article did raise more questions than answers, but that was by design. A lot of the assumptions that go into most tech interview processes don't withstand scrutiny. Interviewing is a hard problem to solve, but we shouldn't be satisfied with the status quo.

>so "(a lot) higher pay" is a moot point

no its not. I know ppl who quit MS and reapply later to get placed into a higher tier after they interviewed there again. Jumping jobs into a higher tier is easier way to get a raise than grinding at the same job.

Just adding I had to do this at a large non-faang corp. Just curious how common this is, it seems like the only work around to strict HR caps on raises.

Jumping ship for a promotion at the original company is a super common strategy in other sectors too.

Extremely common. Interviewing is a tiring and arduous process and companies have all the power. They make the bet that the majority of people are not willing to leave (esp. once they have families).

Very common.

It’s called salary compression or even worse salary inversion.

> a sad indictment of the current state of tech hiring

I've only ever worked as a programmer, so I'm genuinely curious - how does interviewing work for every other job that's out there?

Varies by industry and sector, but there's typically no "technical". It goes: 1) Resume + cover letter + cumbersome online job app, 2) phone screen, 3) On-site, and 4) Reference check.

Typically the interviews are more conversational, with an emphasis on behavioral questions, job history, and relevant skills. Academia and government have their own formats.

Trial shifts are a lot more common in service industry jobs, and I think construction as well.

Biggest difference is references actually matter.

> and quite frankly it's unlikely that the daily tasks involved in the next job will bear a strong resemblance to the kind of synthetic tasks or problems you are asked about in an interview setting.

The blog post mentions in passing that their interview.io site gives users a choice between practicing traditional "synthetic"/algorithmic questions and a different approach focused on "systems design". I'm not entirely sure what the latter involves, but it seems like it could strike quite a bit closer to the generalist knowledge that would be required in "day to day work", while preserving a desirable problem-solving component.

One might even imagine using both approaches in combination, e.g. using algorithmic puzzles as a sort of tie-breaker to choose among candidates who seem about equally strong in the 'systems design' domain.

Algorithmic coding tasks and system design questions are the two types of things you get asked in SWE interviews. They just have practice interviews for both. It's not novel.

None of my jobs have done much to exercise the skills I’d need to get exactly the same job in every way at a place that does whiteboard quizzes, language trivia questions, and/or leetcode stuff. It’s not even about moving “up”, necessarily.