You could also argue that this is fair chance between those who just came to the field.
> On the bright side, it does look like many companies are now realizing this isn't a great way to evaluate someone's ability and are beginning to incorporate more practical exercises.
It's the other way around: many more companies are incorporating Leetcode style interviews as their first "filters" either via online test or quick on-site.
Here's what happened: it used to be the case that only these FAANG companies have the resources and the know-how to either create these "trick" questions or picked a few from their favourite Algorithms book "exercise" part.
These days thanks to many "algo" challenge websites (like InterviewCake.com) that not only provides questions but also correct, detailed, and optimized answers, it is getting easier for employers to just pick a random ones and use that as interview materials.
It /might/ be /somewhat/ true. But I'm skeptical.
Anecdotally, when I started Interview Cake 6 years ago it was already true that most small companies my friends and I interviewed with were using these sorts of data structures and algorithms questions. The handful of exceptions were mostly companies that were outside the "scene" (usually because they weren't in SF or NY).
This was not the case 5-6 years ago.
I wonder how rigorous non-algo challenges can be inside of an hour. Or how you would “get a sense” of CS fundamentals without algo tests.
What are the top level concerns, how do you break that problem down, how might your problems scale, etc. There's a lot of questions I'd expect to get to, and this should be something done along with the team kind of like we're all working on the problem together.
I find it useful to see how well people can talk through the problem, it can lead easily into questions about licensing and rights of reuse, types of errors, etc. If they suggest an approach they've used before, can they explain likely failure cases / benefits? Are there workarounds, detection methods? For example, if you're doing text classification then tfidf+svm is a solid first thing to try, and there's easy ways that can fail which we could talk about.
There's a lot of that you can cover in an hour, and it tests whether someone can explain a potential solution to the team effectively, just as they would have to on a day-to-day basis. We can bring up specific types of problems that we face within it, what we've tried, and we can constrain the problem more or lead someone to starting points if it's a bit overwhelming.
edit - I guess this would fall under some data science fundamentals, but the approach I think works for CS fundamentals. What data structures could you use? What are the tradeoffs? It's not about finding the one optimal solution, but about how to proceed.
This is screening for specific domain knowledge (text processing) not general programming aptitude. That's ok if you want specific kinds of prior knowledge on Day 1 but it is not a way to hire generally smart people.
> I guess this would fall under some data science fundamentals, but the approach I think works for CS fundamentals. What data structures could you use? What are the tradeoffs? It's not about finding the one optimal solution, but about how to proceed.
This is exactly how most algorithm interview questions work.
I'm trying to understand what the OP meant by "real problems" not "academic puzzles". It sounded like they avoided hard algorithms yet "got a sense" of CS fundamentals somehow.
Not really, the domain knowledge here is much more the bibliometrics stuff, which we don't need usually. What I do need is someone who knows that they can't just take any data source they find and throw the latest deep learning hotness at it and call it a day because the F score is over some random threshold.
You can absolutely use this to hire generally smart people, what you are right about is I won't be hiring people who are generally smart but have no basic understanding of any of the types of solutions they'll need to work on and with. Based on team size and where we are, that's completely fine for me right now.
I think this helps find people that:
1. Are able to talk through a problem
2. Understand the kinds of issues they may face, and discuss how to go with that (including business level work, when to / not to use humans, etc)
3. Have experience working on the kinds of problems they are going to face
The tfidf+svm example was not intended as "ah yes, they said the algorithm I wanted" but as a springboard into a further discussion. Maybe they talk about word vectors, whatever, can they explain what the pros and cons are? Where might it fail, or more importantly what would they want to test? Where do we get training data, how long might that take for reasonable quality, how do you measure that, etc.
> This is exactly how most algorithm interview questions work.
The puzzles I think they're talking about are "here is a theoretical problem, find the optimal solution". Like "you have X eggs and need to find the highest floor you can drop them from without them breaking" or the classic "implement a doubly linked list with a single pointer" despite the fact _almost nobody_ would actually implement that.
I'm talking about giving an actual issue they're likely to face and talking through it. Maybe that's "how would you implement user flow for X, given that we've got institutional customers with multiple clients" or "we need to do rate limiting and have X servers, how would you go about that?". For the latter that might go into a discussion on what the problems are with different approaches in complexity, what the cost is of letting someone exceed their rate, of cutting someone off early, etc. Those aren't really my field so maybe the questions are a bit off, but the point is can they contribute to a discussion on the way forwards on a problem which represents something they will realistically face.
The only large company I can think of that matches or even exceeds the offers those companies give out is (surprisingly) Snap. If we go smaller, Airbnb and Pinterest are up there too, but they don't have nearly as many employees or open positions.
Gmafia
Not sure that's true. Just a few days ago someone was commenting here how they were a frontend developer with a few years of experience, yet they didn't know what a for loop was. These interviews aren't about how quickly you can figure out the right data structure, they're about separating developers from "developers".
In my experience, all the questions I was asked were fair game - all of them were about implementing a toy version of a real-life feature (simplified for the time constraints of the interview). All of them felt like "yeah, a competent engineer should be able to do this".
There's no question that a competent, well-rounded programmer should know about these algorithms' existence and when to use them. But whether testing if said programmer can implement them from scratch during an interview... that's a million dollar question whether these questions provide optimal outcomes. Anecdotally, yeah, sure, I did these things. In S/370 assembler at that. But did I use 90% of them since then(and that was when Apple was the only FAANG in existence)? No. So I think the feeling is that this kind of interview tests not so much how good, or experienced, or what not, you are at programming but rather how desperate you are to work at that kind of company to spend a good amount of time memorizing algos and data structures you've learned and forgotten years ago.
Most interviews have 4 sessions broken up in the following manner:
1 hour: coding
1 hour: coding
1 hour: lunch
1 hour: system design
1 hour: cross-functional
In my experience though, the first two are the ones that really make or break candidates, if you don't pass both, you're not getting hired. And yes, most of those interviews just test your knowledge of how quickly you can figure out the right data structure and algo for the question at hand. And even if you know all of the data structures and the applications of each, it can be incredibly difficult to solve the problems in the allotted time.
The other funny thing is that your previous experience doesn't do anything for you other than maybe getting to skip the technical phone screen. Everyone goes through the same process. e.g. https://twitter.com/mxcl/status/608682016205344768?lang=en
On the bright side, it does look like many companies are now realizing this isn't a great way to evaluate someone's ability and are beginning to incorporate more practical exercises.