I have also wondered about the arms race and how it weighs against market forces for the bar going up, especially because as founder of an interview prep platform, I am complicit in said arms race and don't feel great about it (though I think the good of starting interviewing.io far outweighs the bad, but that's a story for another time).
So I looked at the data.
Between 2015 and the first half of 2022, I'd argue that "the bar" was about the same, even though a bunch of interview prep resources sprung up during that time (interviewing.io was founded in 2015, Leetcode was, Pramp was, Triplebyte was, HackerRank was a few years earlier, the list goes on).
Then, the tech downturn happened in 2022, and all of a sudden the bar jumped... because for the first time companies didn't feel like there was an acute shortage of candidates.
Here's some the data about the bar. At interviewing.io, after each interview, whether it's mock or real, the interviewer fills out a rubric. The rubric asks whether you'd move the candidate forward and also asks to rate them on a scale of 1 to 4 on coding ability, problem solving ability, and communication.
We can look at what the average coding score was over time for passing interviews to see how much the bar has gone up.
Between 2016 and 2022, the bar grew a bit (from 3.3 to 3.4). Starting in 2022, it shot up to 3.7. Is this the be-all and end-all? Of course not. But to me this is compelling data that market forces >>> the interview prep industry.
Even so, I do think it's worth considering what someone in your position can do to make things better. If an arms race is inevitable, can it at least be an arms race with positive externalities?
For example, if companies focused much more on security questions during interviews, that would create an incentive for devs to learn about security. Then we'd have more secure software as a positive externality -- in theory, at least.
If we could get companies to ask more questions about AI alignment, that could reduce risks from AI misalignment.
If we could get companies to ask more questions about optimizing the energy usage of apps and data centers, that could be good for the environment.
The pitch to hiring managers would be something like: The algorithms interview is not about algorithms per se. Algorithm knowledge is only somewhat useful on the job. Rather, the algorithms interview is about giving the candidate a chance to signal that they can master technical coding knowledge. It doesn't particularly matter what that technical coding knowledge is.
And, if you ask questions on topic besides the classic data structures and algorithms, that means you're measuring something different:
* You're measuring the candidate's passion to learn CS stuff that's not usually covered in interviews.
* You're measuring the candidate's ability to pick up something new on the fly.
* If you're transparent, and you publicize the topic(s) you interview for, you're measuring the candidate's passion for getting hired at your company in particular.
All of those measurements seem potentially more valuable than measuring how much time they had to study classic algorithms topics.
A common misconception is that Gayle's original book put forth the "right" way to do interviews. Gayle neither invented or encouraged the current interview structure. Gayle discusses the timeline in more depth in a Blind AMA thread you can find online. I think a lot of people are under the impression that books like thes somehow steer the interview process toward this style of interview. At this point, all we are doing is looking at the process as it is TODAY, and trying to help provide transparency and equal information to everyone. We spend several chapters in this book talking about how broken the process is and making similar points to you, but we can't write a book on an interview process that doesn't exist and while Gayle's original book is well-circulated, she (or any of the rest of us authors) doesn't have sway over how big tech companies conduct their hiring.
With that said, I think we are seeing companies start to incorporate other interviews precisely for the reasons you've mentioned. It isn't uncommon for smaller tech companies especially to have a DS&A interview, but also include a system design interview, and maybe even a practical "build something simple like a tic-tac-toe game in front of me while I watch" kind of interview. I do believe things are getting better and more fair over time (remember two decades ago Google was literally asking riddles in an attempt to screen people). I don't buy into the narrative that these interviews should go away entirely (and if they did, it would take at least a decade) because they are still a reasonably effective way to interview people at scale. The Pragmatic Programmer guy actually had a great take on this here: https://x.com/GergelyOrosz/status/1891212829346435103
Here's the problem. It's really hard to get companies to change.
A bit of a long-winded answer that I hope will come back around...
I don't feel great about my role in perpetuating the interview prep industrial complex. It wasn’t my intention – to this day, interviewing.io is first and foremost a hiring company that’s trying to make eng hiring efficient and fair (though bc of the downturn we've been doing less hiring than we'd like). Our goal is to find the best performers, regardless of who they are and how they look on paper, and get them into any company they want.
So why do we do mock interviews?
Mock interviews were supposed to be a way to attract people to our platform, not an end in itself. Over time, though, mock interviews have become a larger and larger part of our business, and that’s part of the reason I wanted to write this book. I'm hopeful that changing the conversation around prep and empowering engineers to learn the material rather than memorize the material will make a dent in an industry that is, as of now, headed in the wrong direction.
The other thing we’re doing at interviewing.io is gathering a massive interview data set. Unlike other mock interview platforms, we don’t tell our interviewers what questions to ask or how to interview. Instead, we let each interviewer run their own process. This means that we end up with a lot of “interview biodiversity” on our platform.
I'm hopeful that, over time, we’ll be able to use that data to do a retroactive analysis where we look at users’ outcomes and which mock interviews they passed and failed to figure out which types of interviews carry the most signal and, over time, conclusively come up with some best practices around the best way to surface good engineers. Because it can’t be what our industry is doing now!
The other piece of the puzzle is having enough mindshare, when we do figure out the answers, to have people listen. We've already blogged a LOT about what companies can do to hire better. Here's our collection of posts specifically for employers and how they can hire better: https://interviewing.io/blog/category/for-employers-how-to-h...
The reality is that employers don't care. They won't change their behavior unless you're somehow making things 10X cheaper or faster. So that's what we want to do, both with respect to how companies hire (making it be about what people can do instead of the brands on their resumes) and with respect to how companies interview (hopefully moving away from toy algorithmic problems that people can drill on and memorize).
It is not about a "higher" bar as much as a more "standardized" bar and more importantly to weed out the "uncompliant" ones.
Leetcoding style preparation is pretty much a 2-3 hour a day over 1-2 months ordeal (given or take). Which means those who are seeking the types who have the time to invest in this repeatedly. This translates your willingness to be part of the time no questions asked.
Worse this is not one off. No matter your expertise/experience level, unless you are doing this everyday for fun, you need to invest to stay sharp. Guess how many family folks actually have this kind of time?
I consider myself a reasonably experienced engineer -both from a ton of side projects for fun and with long stints at faangs + being on both sides of the interview table (note I said experienced, not "great") and i still have to devote 2-3 weeks of ~4 hours each day leetcoding each time I change roles. Would be impossible if family wasn't so forgiving!
Back to your point. These kind of interviews have little to do with bar and more about companies know they can put you through it and to hire those who are willing to give it all up for the co! They don't want (or know what to do with) the next woz. Someone who will follow the rubrics will do.
When Cracking the Coding Interview was originally written, the intention wasn’t to claim, it as a perfect process and to give ideal questions that should be asked. In fact, there are many questions in the original book that I wouldn’t recommend any interviewer use. The real purpose was to shed light on a process that had been shrouded in secrecy for decades, long before the book existed.
The reality is, like them or not, these types of interviews aren’t going anywhere—whether or not resources like mine are available. So the real question becomes: Should the process remain some insider-only system where only those with well-connected friends know what to expect and how to prepare? Or should we make it more accessible, ensuring that everyone has a fair shot with similar resources?
For me, the answer is clear: transparency and equal access for all. It’s not about endorsing the process—it’s about making sure the playing field is level.
I wrote a reply to Aline which applies to you as well: https://news.ycombinator.com/item?id=43145953
It really resonates with me because during my university years, I myself had a poor social network. As soon as I started interviewing, I was like, “WTF is this?!” whereas some better connected peers seemed to somehow “just know” how to be prepared to interview. There definitely wasn’t equal access.
Thanks for trying to make things better!
Is this because of an interview prep arms race, or supply and demand, or what?
Is all of that interview prep, and higher standards for hiring, translating into developers who are stronger on the job? Or at least, stronger when the job calls for data structure skill?