Consequently, I have a lot of successful open source work, but most companies still don’t care and fail me for lack of fake puzzle quiz skills.

It’s utterly baffling to me that companies don’t want creative employees that build successful products out in front of everyone. Where you have a clear view of all their interactions and code.

I’ve tried to change this thinking at my current company and the consensus was it would be biased. Biased?! Code quizzes are highly biased! I’m producing useful functionality to the world!

I believe this and transparency are the new remote work. Some companies will get it early and for others it will take a major upheaval to get them to a place of reasonableness.

I already have to be so focused with what little time I have in my life, and sacrifice a lot, to just make the things I want to make, I just can't imagine finding all the time to do this meta work/training.

Every day I grow more comfortable with the idea that my love of software dev and oss will forever be avocational. Perhaps though, financial instability aside, I'll be happier this way anyway

You can always move onto a more competitive company later once your resume has some solid experience on it, if that's your cup of tea.

Needless to say, I got nothing from it. A couple places I was perfect for the job, but I couldn't spit out elegant quiz code on demand from memory. Sorry folks, head injuries f with your memory. I'll always look shit up so I get it right.

That is encouraging though that not all interviews are like this! I have projected so much into the fantasy of getting a tech interview without ever having one, its easy for me to be swayed one way or another.

Example:

Breadth first search

Recursion

Binary Tree

Etc.

Then it’s more a classification problem.

They definitely are asking puzzle question that an university teacher teaching algorithm would fail. And the only way to is practice stupid question on leetcode

You grind leetcode problems over and over so that you can better recognize these patterns and know when to apply their associated solutions.

The most idealistic scenario is you get a problem you've recently practiced verbatim. Then you just put on an act like you've never seen it before in your life.

But if you get a problem that you can match to a pattern you've practiced, then you're still mostly in a good place.

As an aside, to everyone who has asked me how to get into FAANG - I always say you should be able to code DFS/BFS in your sleep like your life depends on it. 1/5 questions is surely going to be a tree/graph search.

Take-homes get a lot of criticism on HN, but in my experience they're superior for all the reasons you mentioned.

I don't actually see pushback against take-home problems in the real world, mostly just online. When given the choice, most people would prefer a practical take-home over a whiteboard style interview.

For example, it's very tempting for a company to send out tests before doing much resume screening. Why should I spend 30 seconds looking at a resume when I could have the candidate spend an hour on a test, and save myself 30 seconds?

Or what if I set a take-home test which is supposed to take 1 hour, but also supposed to really to stretch candidates' ability and give the best candidates a chance to shine? Probably a lot of average candidates will give themselves a bit of extra time - and before you know it, I'm sending a "1 hour" test that takes an average of 4 hours.

Take-home tests are a lot less objectionable if you can properly guard against these problems.

But for your first point, that wouldn't seem to make much sense given the time investment from the company as well. It takes time to review take home tests, much more time than it takes to review a resume or even for a recruiter to have a quick phone call with the candidate. So why would they send out a test to someone first?

I've usually seen something more like this, which I think generally works fairly well:

1. Initial resume screening 2. Quick phone call with the recruiter 3. Usually some kind of technical call/interview with developer(s), engineering manager or similar, at the very least some discussion of experience and background 4. Send out the test, review it, call the candidate back in to discuss if they want to 5. Probably some other stage at the end

Many interview at a much less aggressive pace than that and I would prefer a take home when I am out of leetcode practice.

Having mentored a lot of college grads and also interviewed a lot of candidates, it's actually extremely rare for people to line up a huge number of interviews in a single week. It can be done if you really, really treat it as a numbers game, but it's not how most people do it.

> Take homes are untenable in that situation as half the interviews would still be a full workweek

Take homes shouldn't be taking a full workweek.

If anything, take-homes make it easier to stack a lot of interviews because you can do them at your leisure in your own time.

> I was already interviewing 4 hours a day with 3-6 people

I think this is the disconnect: You're taking time off of work for live interviews, but take-homes come out of "extra" time. If you're interviewing full time, a take-home is still more flexible than a live interview.

Incidentally, I have the same criticism of TripleByte. Sure, they get you on-site with companies faster, but their process takes a significant number of hours (essentially equivalent to its own on-site), and it doesn't get you out of any part of the prospective employer's on-site.

Take homes are bad because the time investment is asymmetrical. A company can just say to everyone who applies to do the take home test because there's really no investment on their part. At least with an interview, you know the company is investing an equal amount of time into the process.

Triplebyte used to be helpful because it did allow you to skip the initial phone screen but that doesn't seem to be the case much anymore.

What people need to remember is that there's a very real pecking order. Everyone will do Google's take-home but the local business paying local CoL/Prevailing wages? By the time some get to their take home, the A players are already landing in SFO for their final rounds.

The people that are the most hirable won't have time to do 20 interviews, they'll get offers almost right away.

> I think this is the disconnect: You're taking time off of work for live interviews, but take-homes come out of "extra" time. If you're interviewing full time, a take-home is still more flexible than a live interview.

The thing is, with a live interview it's one hour of their time for one hour of mine. With a take-home, it could be 4 hours of mine and... ten minute for them to look at the code and run a test suite on it. So as a rule of thumb, I always suggest favoring live interviews (they are investing more time therefore they place a higher value on the potential hire).

Many of the take homes I encountered also had rapid turnarounds. I would have 48 hours from Tuesday afternoon. A couple of times I was up at 3 in the morning as I had two of them due and had to abandon one of them as I just couldn't stay awake longer.

In hindsight, I should have realized the folly of that earlier, but I was eager to get as many paths going as possible.

Somehow they also seem to get all their offers around the same time to leverage them against each other.

source?

Edit: Oops, sorry, that's not true. 2002 was network. Everything since has been cold.

as an applicant, ive refused takehomes that were auto-sent on receiving the application, but if its after a phone call with the hiring manager or engineer its not so bad. but then, i did end up having to defend why i didnt do a bunch of parts ancillary to the assignment - 'why didnt you build a config loading system?' 'uh cause i timeboxed it and that gives you fairly little signal compared to the business logic'

When I went through the job search, I did close out a few applications because I found out I wasn't too interested in their description of the role (after a conversation with more details than that was in the job posting), or simply if their screening process wasn't worth my time. The one I lamented about pass through those interest tests as the job genuinely fascinated me.

A take home that is 1-2 hours (what might be the length of an in person) is fine.

A take home that is 10 hours of work on the other hand

1. They do not scale horizontally. You do a takehome and get rejected, then your effort has become worthless when applying to other companies. Versus grinding leetcode - it scales across a huge breadth of many different companies.

2. Often times a takehome is not. instead of an leetcode interview. It's in addition to the leetcode interview. So you end up needing to practice leetcode anyways. Now you just have more work to do - work that doesn't horizontally scale.

Coincidentally, I've started working on a product to help this[2]. The general idea is that tasks like pull-requests, bug fixes and realistic take home projects are a better way to test candidates. Everything is done using Github, the tasks are all realistic and candidates can use their own laptop and development environment. It's not fully ready yet, but if anybody wants to chat about it just use the info@ address for the site.

[1] https://www.sciencedaily.com/releases/2020/07/200714101228.h... [2] https://devscreen.io/

Expecting a random hire to already be familiar with your deployment process and tools seems very strange. I've done deployments in many situations and, well, I wish there was an industry standard. But as you point out there are many conflicting tools and using them is never turnkey.

But if a company did leetcode questions without the interviewer in the room, I find it exceedingly unlikely that people would shout "Interviewing has been saved! It now tests skill rather than anxiety!" As such, the conclusions of this paper don't seem especially relevant to the discussion about interviewing processes.

No offense to anyone, but it strikes me a little bit as armchair expert hot takes saying "why not just X". Personally I think that rather than complaining about leetcode-style loops, it'd be more productive to think in terms of what the problem domain is: namely, many of these companies are typically large and want some level of standardization in the hiring process: you have to contend with things like some ethnicities being culturally prone to "hiring down" to inflate subordinate count, talk-the-talk-but-dont-walk-the-walk types, thinly disguised sexism/ageism/other biases that are excused under the "cultural fit" umbrella, mismatch in terms of desire to provide technical interviewing skills vs volume of candidates, etc. It's easy to attack any one methodology simply because if such a thing as a silver bullet existed, we'd all already be using it. All interview methodologies are flawed, and for better or worse, leetcode-style happens to predominate interview loops because it has a semblance of standardization (though ironically, IMHO a good interviewer ought to be looking between the lines more than seeking specific answers, and a lot of leetcode-style interviewers don't get that nuance)

If anything, my two cents is that leetcode ain't going anywhere, so the next best thing we should be focusing on is making it more widespread that memorization culture isn't and shouldn't be the evaluation criteria (this goes both for the interviewer and interviewee sides)

I've been doing this long enough to know that the things that matter in every job at every company are almost never technical ability. It's skill in managing deadlines, collecting requirements and dealing with changes, divvying up work with teammates, things like that. Things that are difficult to test for, even on a whiteboard.

Asking questions about experiences and posing hypotheticals to see what gets candidates interested, or what stories they have to tell, these have been a better predictor of eventual success in my experience than any sort of technical test.

Agreed 100%! I'd love to explore scalable ways to test for these. I've heard of people in smaller companies who get lots of success with non-leetcode style interviews because they can either have full control over the interview cycle (e.g. a hands-on CTO) or a still tight-knit "rock star" team that "gets it", but the challenge is that you end up getting a huge amount of variability as a company scales to hundreds or thousands of software engineers and things like rush hiring to meet deadlines start to enter the picture.

I like to start my interviews asking people about recent work experience and surprisingly that has been a pretty good indicator of how well they'll do in the technical portion. It's still a bit challenging to evaluate because experience doesn't necessarily correlate with potential. I've had a number of candidates whose description of past day-to-day work was nothing to write home about, but you can see their growth potential by really reading between the lines.

Posing hypotheticals is actually relatively standard practice at big tech: it's called the system design session. IMHO, you can get a lot of good signal from these if you do it right.

- generic HR chit-chat. Our HR rep was really good at uncovering personality problems (and she was used to dealing with devs :-). Every single person she raised a red flag about also had issues somewhere else in the process and we learned to trust her instincts.

- What did you do at your last job and what did you like/hate about it? What code were you most proud of? If I asked you to design a system to do X, what would you need to know, what would you be concerned about?

- Basic SE theory: what do you know about SOLID?, what are some concerns when dealing with shared data in a multithreaded design, etc.?

- Programming/design exercise with the interviewer as a co-worker/resource. Done on whiteboard, or sheets of paper or whatever candidate is comfortable with. Code does not have to compile, no online access allowed.

There were scripted questions at each level (except HR) and the interviewer keeps notes on the candidate response. OK to dive deeper or go off on tangents as needed. At the end of the day, or worst case first thing the next day (so the interview is still fresh in everyone's mind) we'd meet to compare notes & vote thumbs up/down.

We had a known problem set that all candidates had to work and this consistency kept HR happy (although it was an uphill battle to convince HR that it wasn't a "test"). The problem was sized so it would be just barely possible to complete the entire thing in 45 minutes. In fact, I think in all the years we did it, only two people every completed the exercise, and one of those we suspect knew about it ahead of time.

However, the problem wasn't a blind "do this on a whiteboard" exercise. We didn't really care whether or not they finished: rather we wanted to see the process they took, the clarifying questions they asked, where they got stuck, how they asked for help, etc.

The candidate could choose to work in complete silence, or work with the interviewer as a team exercise, or a mix of both. The point of the exercise was to see if they could (a) program their way out of a wet paper bag and (b) do so in a way that didn't completely alienate the people they would have to work with. Leetcode? Obscure algorithms? We couldn't care less.

It was surprisingly effective.

Ideally one should not need to check an entire barrage of checkboxes perfectly, but enough in some dimension to "prove" they aren't completely clueless about programming, and also not raise a bunch of red flags (e.g. if you tell me you have 10 years Java experience and then struggle with class syntax, that's obviously going to look bad).

Ideally there are peripheral questions where one gets the opportunity to display well-roundedness (I like to ask about testing in an open ended fashion, for example).

Ideally, an interviewer org ought to take a bit of effort to cater questions to roles/stacks more appropriately. Web folks tends to do this well IME.

Unfortunately, many interviewers take coding/algo sessions as an opportunity to do a pissing contest either because they haven't been trained/prepared well or because of some dumb sense that company brand is somehow related to their personal worth or whatever, hence the bad rep.

Sorry, what? Which ethnicities are these?

I think answers like the parent are why Google started doing "googliness" interviews.

This is the redacted exchange:

Hi [PeopleOps], I started the assessment and was very disappointed to the point where I didn't complete it. I've been an Engineering Manager for about three years and still have a lot to learn, but I've never encountered anything that looks like a set of brain teaser puzzles & psychometric analysis similar to this test. I have to interpret this a huge mismatch between how [Company Name] defines the best skills and aptitudes for the role and my own, so I'm going to withdraw from the process. I can't imagine a lot of candidates are prepared to jump through these sort of hoops but that's just my single perspective.

MONTHS LATER

Thanks for the time earlier this week.

I wanted to give you a heads up as to where we are at in our Sr. Engineering Manager hiring. We had an Engineering manager join us earlier this week and as of right now, we're ok on the hiring front.

I'd love it if you could provide us some feedback on the process as we're always looking to improve. Here's a link to a very brief survey that would really help us out. Thanks!

The issues that candidate employers are concerned about are (should be) more along higher level constructs such as ensuring quality or how to decide on a new technology or platform. What's a really interesting bug you found -- where really interesting probably means it was a system level issue, not just some bad code.

The interview becomes a detailed discussion between peers who are each trying to understand if they want to work with the other.

If someone started out with leetcode crap I'd probably react exactly as you did.

I'm a script hack and sysadmin/SRE, not a software puzzle guru. I look up the syntax I need for the language I'm writing today. I may end up programming a different language next week, month or year. It's not worth the trouble memorizing the triviata of every language I ever used.

There are only so many ways to cast variables and write loops. Everything else is nomenclature and punctuation.

Ask me to address a system problem, don't give me a BS puzzle.

In general, I dislike the interview process too, but there just isn't a better way to do these, as is clear from this large thread. If you look at it from the company's perspective, you have to conduct interviews at scale, and it can't hand off the same system problem to everyone.

You're being tested on your problem solving ability. It doesn't correlate directly to your job, but the general assumption is, if you can solve puzzles after studying for them, the chance that you can solve any random problem thrown at you is high.

How do you know this? The signal of them dumping you with some leetcode should be an indication that it wasn't meant to be?

But a new engineer was assigned to handle interviewing and it was all about a couple of leetcode problems he found on the internet.

The HR person was a complete dolt though, and didnt even understand how to interview me properly...

Sucks - I still respect the company, but their recruiting method was piss-poor..

My prior company would have interns do leetcode and system design. Intermediates and up had some paired code writing and a far less formal system design discussion.

Most of the team might have been hired through a network and you only get leetcoded as an external unknown.

The job description was top notch. It was a clear description of what they needed this engineering role to do. Inspire of it being a bit of a niche, I had almost perfect set of experiences from my prior jobs that made it very interesting.

Fun because it would have given me an opportunity to _improve_ upon my already bootstrapped skills. Productive because, I already have a head start than another engineer getting into the groove of things. This is HN, I can be tech specific :-) I have worked a lot on performance from HPC perspective. The job was at a High Frequency Trading platform that needed a sys.engineer who would work on extracting performance with things like pinning a CPU core to a process and several optimisations at the OS and kernel that would shave off a few milliseconds in an already fast application.

I recently did a round of hiring for my team, and we had several excellent candidates, but could only hire one, so we made a choice.

It’s always sad (for me) because I tend to see the potential in almost everybody, and wish we could hire all the people.

But the enemy has the high ground at the moment.