If all your career experience so far has been at the NSA, yeah, you might want to do a side project before looking for work.

So I can't say what the code specifically does or who it's for, but I can talk about how I worked on a c++ engine wrapped in a Java server that was responsible for coordinating a large group of non-homogeneous assets, as well as various architectural details (what libraries did we use, database, messaging setup, etc). So it's not like I have nothing to talk about in an interview, plus the fact that I can't get too specific adds mystique. It's kinda fun, because in my experience there's an assumption most engineers/engineering managers make about what I've worked on from that first statement, particularly if they know where I work, and it's actually not that. :)

So I can talk about architecting a system, about customer feedback and redesigns, but I can’t talk about work I did that will span another three years.

I think at FAANG it’s usually fine — most people have good enough examples even without their full repertoire. But at mid enterprise or F500 companies I could see this being a bit impactful.

Sorry, to be clear, I mean can you give an example of something you couldn't use as an example, but is now public.

I've worked at GAMMA, and agree, it's very doable to turn my work generic and talk about it in-depth, compare it to alternatives, etc. so that was the main driver in asking for an example.

if you have been through a CS program you have worked on probably at least 10 projects. and then there are your personal projects. if you have any experience you can talk about at least some of these apart from the stuff under NDA.

so, this approach is much better than timed coding interviews and is a much more fair system as it rewards and takes into account experience.

That's fine for a new graduate, but many of us are decades out of college. What we did then has almost zero relevance to jobs we might seek today.

Here's a basic straw man rails controller. There are a few things wrong with it. Apply suggestions. It was nice.

Maybe I started a new compiler or something at my company but didn't have the chops for it and the project flamed out. If I lie and said that all my goals were achieved and all the hard technical challenges were overcome, how can you tell?

Or maybe the project did succeed but someone else came up with the idea and led the efforts. I was there for the technical discussions and grilled the lead on why he made the choices he did, so now I can answer your questions and sound like I know what I'm talking about.

Software engineers can make a lot of money, the incentives to game the interview process are high and people attempt it often...

Everyone you talk with, has probably one or two anecdotal stories of such. "Yeah I worked with this CS grad that couldn't even write FizzBuzz" - yet we ignore the hundreds of other that do their work just fine.

And this is fought with setting up ridiculous 8-part technical interviews where you'll have to whiteboard some leetcode questions ("Given a problem, show us a working O(Log N), or preferably O(1) solution. You have 45 minutes") or design questions ("Show us how you would design slack / discord / zoom / etc.") that drags over 6 months.

For someone to be truly incompetent, or even not good enough to meet the company standards, the current system is overkill.

I asked him to explain how to do it in 2. And he was like, “what? It’s easy!” Then another person on the team who over heard it also asked.

Eventually the whole team was there, asking him to prove it, and he spent over an half an hour trying to explain it, and none of us were getting it. The consensus from everyone (except the manager) that it was an unfair question that relied on a a very subtle assumption about the data that wasn’t obvious at all, and that 3 passes was the minimum required Without the that assumption.

Of course the manager said everyone was stupid and it was a good question.

That manager remains my counter example of what a good manager is.

A friend of mine at a FAANG recently dealt with a bad hire. Their guess was that in the 6 months the bad hire was there they cost the company maybe ~5 million between wasted engineer time and delayed release schedules after people kept having to put out their fires. On the other end of companies, I've heard of seniors corrupting the database and its backup and ending an entire startup.

This is not a defense of whiteboard interviews per se, just an observation on why companies desperately want to avoid bad hires.

I've worked with one or two of those, but I've also been on interview loops that rejected candidates who went on to massively successful careers at a different FAANG.

The cost of false positives is relatively easy to quantify, the cost of false negatives significantly harder. Which is higher?

I'm always curious how people interview doctors. As bad as getting a bad programmer might potentially be, getting a bad doctor must be exponentially worse, so you'd think they'd have vetting the unqualified folks down to a science by now.

Mistakes can also be career-ending and potentially result in criminal prosecution.

It's way harder to get hired as a doctor than "do a bit of leetcode on a whiteboard".

I'd imagine software/IT might be only place where people who have delivered a couple of toy sized, half-assed webapps are now experts commenting on 'engineering challenges' and 'industry trends' with profundity.

They go through several years of medical residency under supervision before becoming a full fledged doctor and are burdened by expensive liability insurance and ongoing education requirements. That takes care of the competency beforehand.

It's not material to my point either way. I was only arguing that for some company's being very afraid of bad hires may be sensible and dismissing their concerns as hysteria does not seem obviously correct to me.

I think he was trying to point out that if some of the bad candidates had passed a whiteboard test, it brings in to the questions of the effectiveness of whiteboard tests to filter out bad candidates.

I remember a guy we hired years ago who answered all the tech questions we asked with flying colors. He was lazy and unmotivated and ended up dumping all his work on other people (me). I would have much rather had a motivated person who I had to teach some stuff to rather than him.

Who said that?

Others in the comments have expressed some legal concerns about "discussions" instead of Interviews, but I feel this is a hyperbolized fear based off a laypersons interpretation of US hiring law. The law requires you ask the same set of questions of all candidates, yes, but it reasonably understands that questions on past projects/work of course will never be the same, and these are legal and valid areas to go off-script.

So suppose in your situation you have a candidate and they talk about the project they worked on. To hear about their contribution, you ask (as you would all candidates):

"Tell me about a problem you encountered on this project that was particularly challenging from a technical perspective"

The candidate will tell you and likely their solution, and then you can start with the why's and how's.

"Why did you choose this approach? What are the benefits?"

"How did you arrive at this conclusion?" (you can further restate the question with "explain how you got to the idea that you needed to get here")

Make it a hard rule for yourself that you must be able to build the entire timeline and understand the candidate's role in that timeline. I find that even if it turns out they mostly were receiving orders from a more senior resource but could reasonably explain in their own words now why they ended up doing what they did, this is still a good candidate for me as they used available resources and took away a good lesson. If there is ambiguity, you can even just ask "why do you think your senior colleague had the right idea here?"

The idea of a discussion versus a checklist is you want to learn how a person thinks and how they approach issues. Even as a follow up to more factual check-list questions, "why" is very important as it helps you to understand the current state of the person. Just be honest with yourself that not everyone will have the same background and be honest on whether the "why" is relevant for the position. For example, some trivial linux kernel knowledge I would consider "nice to know", but it's by far not essential unless they're doing specific dev work on that field or claim to know it.

The other big catch you need to train yourself for is accepting that people will do things far differently than you do, even things that you consider wrong, but they work. The important part to focus on isn't about how close their answer is to yours, but to make sure you understand how and why someone reached that conclusion.

Why and how are very safe and legal questions as a follow up to a very straight-forward technical question.

Does it? I have never heard of such a requirement in a law (though I’m not a lawyer). That might be an employer’s decision on policy they use to ensure compliance with the law, but I couldn’t quickly find any law requiring the questions to be uniform across candidates. (It’s also a difficult set of terms to quickly and confidently exclude the possibility that one exists.)

I guess that’s a longer (and hopefully more polite) way of saying “citation, please.”

https://www.eeoc.gov/employers/small-business/3-im-recruitin...

Screen applications consistently. Apply the same standards to everyone applying for the same position.

Effectively, you should have the exact same criterion for all candidates for the same position and avoid "on the fly" questions/too heavily customizing it.

The legal discussion on all of this has basically boiled down to that you must ensure you're going through the same process for each candidate with the expected variances based on their employment/personal history (as it's relevant to the position)

I may overstate the situation slightly with that statement, but the idea is that if you ask about a networking stack for candidate A, you should ask the same probing questions for candidate B also. (Consider maybe you know that candidate B doesn't know this stack and will look "negative" on a review to the hiring managers, but you really like candidate B for other reasons)

Ultimately the idea is less about the specific questions and more that Candidate A and B are both reasonably considered in the same fashion for all of the requirements of the position, and that you aren't adding/omitting elements that may influence the applicability of the candidate in any fashion.

So let me restate: It's not required you ask verbatim the same words for each candidate (this is the lay-person over-read).

But for a given position, it should be established in advance a series of competencies that are necessary for the position and that are to be asked of each candidate. How you go about investigating it may vary from candidate to candidate, but should two interviews be reviewed, it should be expected that both interviews cover the same topics to a reasonable degree. (e.g., it is reasonable that if you ask "have you ever worked with library X" and the candidate flat out says they cannot answer questions on library X, you have reasonably covered that topic with this candidate. If you ask another candidate and they do have experience with library X and you ask some additional questions, you have not violated the spirit or letter of the requirement in this case.)

But, if you wanted to ask the recruiting team to add an interview to get more feedback, this would be a clear no.

Some of the best candidates I’ve been involved with hiring have been compelling for reasons you’d never get to in a very structured way.

For instance, a candidate who was just out of university. They were very shy about technical topics, and their tech depth wasn’t great. They went to a university in Africa that I knew taught people more stuff that was wrong than right, and they were starting at a disadvantage. To get him out of his shell I asked about things he’d done outside of academics. He told me that he had not done much because he was very busy. Busy with what?

Oh, just starting a real estate investing side business with his family where he’d been responsible for their international expansion into the UK. With no finance background, he’d spent his time understanding the financial, legal and tax implications of their expansion and successfully launched in his third year of university.

And I’d done a bit of finance so we talked about concepts there and he was able to walk me through complex topics in simple terms.

My job for that interview was judging bias for action, curiosity, and diving deep into metrics. He couldn’t show that for tech because he went to a horrible university and had been very busy with other extra circulars.

He apologized to me for spending so much time talking about non tech subjects. And here is the problem: people who are bad at interviewing are punished by rigid processes. There’s a lot of data to show that women and immigrants from certain backgrounds are more reserved on average. You have to be a much better interviewer to pull off a structured interview that lets these candidates shine.

Stripe handled this by having an extra interview baked in by default. People who were viewed as no risk hires didn’t have to do it, but the default was an extra interview to cover risks. Because it wasn’t an aberration, it wasn’t giving a candidate preferential treatment.

I honestly think it’s one of the smartest interview loops I’ve seen. I think they are one of the few large companies gambling a bit on the false positive side of things to shave off false negative points. And honestly, I was impressed with the people I interacted with at stripe more than AWS. They’re doing something right.

Making definitive statements about liability is not trivial. I can confirm that at some very large companies legal has banned this style of interview citing discrimination concerns. I'm not a lawyer, so I can't comment on whether they're call is more or less correct but it certainly doesn't seem settled.

Also working for a "large" US company and discussing with peers from others, no such banning exists. There is very clear training and a heavy restriction on who can perform interviews, but this is not the same as outright banning it.

I do a coding task first and a Q&A afterwards — because these are the requirements.

I did many interviews. My experience is: I could skip the Q&A completely. Most candidates could answer the questions just fine after reading the Wikipedia article for 10 minutes. In the interview I can see if they already read it or not. But I don't think it matters.

What matters are the coding skills. The coding task is quite simple and half of the candidates (with master degrees and 'years of industry experience') fail. But these candidates are often good talkers when they are chatting about the benefits of agile methodology and so on.

I totally agree with you that a candidate should just write any program. I believe I know after 15 minutes if they are developers or not.

But I also agree with the idea that we interviews must calm down the interviewee and remove the nervousness. I don't want to see if they stay cool in an exam situation because daily work is not an exam situation. I want to see if they can code when they are relaxed.

If half the candidates with a master's degree in CS are failing your test, that should be a pretty huge red flag to you that your test might have an issue. What question are they failing?

C++ is a particularly badly affected language for some reason. I think there are a lot of people who learned C++ once, a long time ago, then moved to Java as quickly as they could when it came out. But nobody likes to admit that maybe they lost a skill they once had, so they keep putting it down on the CV regardless. And then when asked to write something like hello world, they don't remember how to use std::cout or whatever.

These guys are applying for C++ dev jobs but are completely unable to write a program. Most of these fake programmers have studied electronics (not CS) or have studied in a country where you can buy a degree.

What are you actually inferring from the situation - how well the candidate can write software? How well they tackle stress and anxiety? How much spare-time they have to grind these types of questions? etc.

Probably a mix of them all, but there's a lot of irrelevant noise - if you're just looking for one thing.

As companies ask 'Leetcode easy' questions, there came to be thousands of online blogs, youtube channels etc.. which trained even the n00bs who can't write good code otherwise. If they train very well they can solve 2sum or write binary tree level order traversal without understanding much. Of course not all interviews can use novel exclusive questions, and these have a non-negligible chance of passing.

Now if you are hiring, you would think, "If these n00bs can solve Leetcode easy with practice, the good ones are solving Leetcode medium with same level of practice. So let's raise the level of questions so that we don't end up hiring these rote-learning noobs". And it continues.

I don't think there's an obvious solution to this, if you don't want to lose the statistically good heuristic of problem solving skills, in order to weed out candidates.

If you interview for an R&D position at (e.g.,) a biotech company, in academia, or one of the National Labs, you're usually asked to prepare a 30-45 minute presentation about your past work. New grads usually use their MS/PhD thesis defense slides; other folks often have a conference talk they can expand. Some places also do "chalk talks" where you describe how you'd approach a new problem of your or their choosing.

This presentation is the jumping-off point for the rest of your interviews. If you talked about building a compiler, someone is going to ask for details about the lexer, maybe have you do implement a very simple tokenizer on a white board. Another person will ask how you measured its correctness/performance. Yet another may dig into how you organized the team doing the work, etc.

Maybe, as you propose, someone else helped with parts of the work. This wouldn't necessarily weed that out. Nevertheless, I'd argue that being able to justify the decisions that were made--and cogently explain when/why they might be different--isn't actually BS; it's understanding.

The point is to have a conversation about a project that the candidate understands well and is passionate about rather than asking them a bunch of questions that we already know the answers to.

Before the interview we review the code base and try to understand what it’s doing by the documentation provided in the readme. During the interview we get the candidate to demo the project and any questions that came up in our code review we ask at the stage of execution in the project demo. The interview lasts for 2 hours and we’ve had several rounds of candidates put through this process.

Both we as interviewers and the feedback from candidates has been very positive. The interview ends up being a day-to-day normal experience within the team and this really helps us to gauge the team fit. I think this helps to hire people that compliment and expand our skill set rather than hire people who are basically ourselves.

Contrast it to giving a candidate some slightly broken code in a framework related to the role and then asking them to a)fix it and b)implement a new feature of their choosing and document it.

The advantages of the latter approach:

* The interviewer doesn't need to prep beforehand as they already know both the problem and the codebase. This means they can ask much more interesting questions and don't have to invest significant time in reviewing a project that might be in a field in which they have no experience. This in turn leads to better discussions which in turn leads to better interviews.

* The candidate is given a much tighter problem definition and isn't required to come up with something a)novel, b)not covered by their current employer's NDAs.

* The candidate's time is respected because they can be told how long it should take them up front.

* Each candidate gets a standardised problem and so it's easier to compare between them.

When I give take-home assignments, I'm just looking to quickly confirm that someone has a working home dev environment (i.e. they don't just code inside environments that other people give to them) and that they can understand a small codebase and write clean code and document it. Everything beyond that I can find out by talking to them during the interview.

When I actually used this technique in interviews it was interesting to see how many supposed senior engineers would reply with things like "I'm getting a %JAVA_HOME NOT FOUND error, please can you fix the repo and let me know when it's ready for me to work on?".

The test is intentionally designed to filter out candidates who cannot meet or do not want to meet the technical requirements.

Yeah, no, nobody is or should be giving you their own personal work. It's offensive and probably illegal that you'd even ask.

> The novelty and creativity of your submission

Yeah, no.

"Passion projects" in my "spare time"... maybe once the kids have grown up and flown the nest...

Leetcode is easy... I memorise a bunch of stuff, do the dance and pass the interview. If i'm lucky I get a problem i've not seen before and actually have to use my brain during the interview.

I took a look at leetcode when I was interviewing and decided it was a waste of time for me to learn that dance. I was happy with my chances with the companies that didn’t use it in their interviews. And it worked out fine.

Side projects will help you learn useful skills and it is the suggestion of OP that it should be used by more companies in place of leetcode interviews.

Btw, a lot of companies don’t use leetcode for hiring. In my last job hunting season, I would guess than less than 20% of processes used leetcode. Pretty far from “most” companies.

The leetcode dance is, at least for me at this point, much lower effort than starting a side project on the understanding the code I produce will be reviewed during interviews. It's like Sudoku, once you've done a "few", you get to the point where you're able to solve them quickly.

The nice thing about leetcode it is easy to bound the time to what you can handle. I do one puzzle a week and set a timer for 20 minutes. Then I get the answer and browse the forum. It's basically the equivalent of solving the Sunday crossword for me, and it keeps my algorithm skills sharp. Probably no worse than burning a lunch hour on HN.

True, but would your employer care (if they found out) if you copy/pasted a small portion of code that’s not considered critical IP (like util functions, and an integration syncing records from your backend the Salesforce, or something tangential to the business outside the core product)

May technically be breaking your employment agreement, but I can’t imagine it would be too hard to pluck out a decent amount of code, and re-write portions of it if necessary to “anonymize” it for interviewing purposes.

Or if you happen to be interviewed by someone like me, my approach is simply “if you can’t show me the code, show me the UI and explain how the backend / frontend works, and a discussion ensues.

Who decides what is "critical IP" and what is not?

This is a terrible advice, and no, you should never copy code from a work project without being pre-authorized to do it.

It's not just common sense, but you also might trip up over unexpected legal issues (licences etc) that you may not even be aware of.

What? Yes. I expect I'd be in prison for a few years. Your interview approach selects for people who don't follow their NDAs

Sergey Aleynikov went to prison over copying GPL code.

To add some context, my work is in Financial Services, trading systems and whatnot.

I've found this technique to be extremely successful. It's possible I may have had some false negatives, but I've never had a false positive. Everyone I've recommended for hiring has been successful.

It's possible you're good at spotting good people, and rejecting bad people, but it's also possible that hiring is just easier than you think it is, and most people are capable of doing the jobs you hire for. You have no way to tell if you'd have the same result just hiring people by picking random resumes. Maybe negatives are just rare.

This is the impossible problem with hiring. Every interviewer wants to minimize false positives (they're expensive!), but every interviewee thinks they're a false negative.

I don't think this could be true? If you think you don't belong/deserve the job (imposter)... why would you think you deserved the job (false negative).

The fear of BSers seems to be what holds people back from this approach though, and I can see that it wouldn't work for certain non technical fields.

That's just survivorship bias waiting to explode. Also, they're successful by some arbitrary metric that is applicable only to your company.

I've been on the hiring team for every employer I've had for the last 15 years and have hired dozens of people.

I also just generally do not think that most people can even put together a 2-3 hour presentation of their past work that goes over well. Most of the time people can't really talk more than 30 minutes about past projects. That requires a whole different set of skills, which there are certain contexts where I'd value that more highly. But my initial impression is that if we set up our interviews this way, we'd also end up filtering out a lot of people who'd be great, since not picking the perfect project to demo basically dooms the entire interview to be a flop. With multiple interview types, you increase the chance that there's an interview they really shine in.

My experience has been that a lot of candidates can't even fill 45 minutes. I ask "what was the most interesting part?", "what was most technically complex?", "what would you do differently if you did it again?", and they just don't have nontrivial answers.

Yeah, because you can have a career where you're just gluing stuff together to make business apps for, usually, simple business problems. You're looking for craftsmen but you're interviewing plumbers.

Most people look up the local business listings, pick anyone advertised as "plumber", and call to make a service appointment. Or they use a general contractor that already has a list of approved subs. Master plumbers don't have to answer little trick questions about brazing copper or about finding lead pipes in an old building. People somehow trust them to know what their job is, and do it.

Rarely, one might encounter an unreliable plumber. They might not get paid, and any other plumber is usually able to fix their botched jobs without hurting the budget much. Review sites exist to track building-trades business reputations.

But the analogy breaks, because no one trusts software and IT folks to do their jobs competently. The default assumption is that we are all know-nothing hacks who could destroy the company with one keystroke. All our knowledge is assumed to be tightly siloed, and does not transfer between similar technologies. C++ people can't do Rust or Go. Java people can't do C#. Desktop people can't do the cloud. Back-end people can't do UI. CMMI people can't be Agile.

It's madness.

There are a million reasons why this approach doesn't work. Maybe they just don't like talking that much. Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

None of the above are good reasons to pass on a candidate.

Not really. If someone can present solid reasoning and argue their point well, they don't have to think like me. Three's more than one way to skin a cat.

> Maybe they don't like the technology you're discussing. Maybe they just don't care enough to have an opinion on it.

You mean the technology they are being hired for? Yeah, hating, not caring about it is probably not a good motivator to go for the job then, wouldn't you agree?

> None of the above are good reasons to pass on a candidate.

Beats the whiteboarding. I was hired like that in my last 3 jobs and I have used the same approach myself with rather consistently good results.

Thanks for sharing - hopefully a lot of people will use your anecdotal evidence to impact the way they hire people too!

I've learned the hard way to be very reserved when giving opinions about technology X because interviewers sometimes get really defensive about the thing they like about technology X.

Also, putting the emphasis on the interviewer understanding the candidates code instead of the other way around is never going to be popular ;-)

It's also hard to scale it, make it objective, and keep the efficiency high (most developers prefer not spending their time on either side of the interview table)

There is another advantage to the "algos". If you can learn algorithms then perhaps you can learn other things as well.

Every new job I've had involved quite a lot of learning in a very short period of time.

Learning whatever language, and whatever standard library is almost always trivial. They're usually not all that different, well documented, some with textbooks even.

Learning to navigate and reason about the huge number of undocumented, often arbitrary, system architecture decisions, design decisions, code layout, etc. etc. that make up real code bases.

That's hard. Really hard.

Show me something you've done that you're proud of. Take me through it in depth. Answer some questions on it.

Doesn't have to be a free-time project. If you don't have a free-time project and you're too NDA'd to discuss previous work, then present some language feature or something.