Surely this will filter out ~50% of people who are good but don’t have any public code?

I have a family and as such no free time for coding so I haven’t written any code I can legally show anyone else in more than a decade. But everyone who has employed me is more than happy with my work. Not to mention code is only half of why you would want to employ any developer.

I agree. I would be happy to provide references of the developers and engineers in my ex-teams, whether collaborators or mentees, who would give me glowing references.

Instead, I'm asked for senior-managers/directors as references... who were the reason why I left in the first place. I guess HR just wants to know I was actually employed there (tick box)

> Surely this will filter out ~50% of people who are good but don’t have any public code?

Not OP, but I follow a similar process when I have to do coding interviews. I work around the "no public code" problem easily: "great, then pick an open source library you use regularly and let's go through and look at some of the things you do with it, what you like about the API design, and some things you stumble over or wish were better".

I've had candidates go through everything from jQuery to D3 to Spring to just parts of the Java SDK.

Also, in my experience, the percentage of people who have _zero_ public code is small. Maybe 10%. Certainly not half.

> pick an open source library you use regularly and let's go through and look at some of the things you do with it, what you like about the API design ...

People who have plenty of time to prepare, eg carefully study and think about that API before the interview, will tend to do better in the interviews.

Although they aren't necessarily better at coding.

Meaning, you're slightly favoring people with lots of spare time. Not saying that's a good or bad thing, just maybe something to be aware about.

Also, what open source project they happen to choose, will matter I would think. That's a bit like tossing a coin as part of the process

>Surely this will filter out ~50% of people who are good but don’t have any public code?

It then becomes a take home assignment where you get to pick the topic.

Surely you wanted to protoype some tech but didn't have the opportunity at dayjob - so make that prototype and bring it to review.

Much higher interview value than take home assignment IMO, but you need to be competent as an interviewer to enter into a discussion on something you might not understand yourself.

And it's also higher value to the person applying because they get to protoype stuff they find interesting.

Work created in 1-2 hours is very different from a work created in 1-2 months.

I don't see myself reviewing your 2 months of work at an interview. Showing me a prototype of something you thought was interesting and walking me through, pitching the tech and debating pros and cons would be enough for me to estimate what it would be like to work with you and my estimate of your experience level.

Yeah, but at this point you're not left with many other options other than sticking your finger in the air and hoping you can detect if they know how to write code.

Only if you are limiting yourself by the options mentioned by the top comment author :) I was interviewed many times without showing any of my code. The most effective way I saw: the interviewer gives you some code and asks you how you could refactor it, what parts you would implement differently.

To be fair, my "interview" for Fastmail was debugging a real production issue in realtime with one of the founders.

Hi, comment OP here. I decided to blog about this. It's cheeky, apologies. https://siliconvict.com/articles/6-how-to-hire-actually-good...

You do you.

I can't see how your approach can scale to hire tens or maybe hundreds of engineers.

Sounds like something that may somewhat work for you when you're the only one doing the hiring for your small team of buddies at a startup somewhere.

You're not even trying to tackle things like hiring for multiple roles, interview fairness, subconscious bias, technical diversity, feedback, or making the interview process accessible to a larger group of people that don't fit your cookie-cutter from the get go.

On a different note - I really hate the tone of your post. You come off as an arrogant person who's full of it and "has it all figured out". I'd easily pass on you just by reading these posts.

If you need hundreds of engineers to solve a related set of business problems[1], I wager you've done something wrong.

[1] and if these problems are unrelated (think Uber's human driver app vs self driving), then why are candidates passing through the same committee?

There's a difference between hiring your 2-3 buddies that'll work with you on your toy startup (many of which ended up shutting down fairly quickly, reading OP's blog) - and actually staffing a diverse team that works on a complex product that's used by many clients and is held up to a high standard.

Mature products require competence in areas such as security, architecture, compliance, devops, research and much more. Hiring has to reflect that and account for it.

It's silly and immature to suggest a 90-minute chat about some crap your candidate will bring with them is somehow the answer here (and claiming that if they don't have anything to bring with them, then chances are that they're worthless anyways and you shouldn't waste your time on them). Especially without backing this up with any meaningful data (ie. attrition, number of people hired, actual impact on business, feedback received, etc).

No worries at all :)

> Sorry to be blunt, but chances are they aren't very good, and they are definitely not as good as they think they are.

I don’t know what to tell you.

It’s just so wrong it’s kinda wild how sure you are about this. At least half of the excellent colleagues I worked with in the past 10 years don’t fit your criteria. You’d be lucky to hire any of them.

I agree with you, this cannot be regarded as a healthy approach to hiring and comes off as delusional.

At the same time, it might sort of work, just that since they a bit randomly discard a fraction of the job applicants, the company will end up paying more, for those it hires (supply and demand). But maybe the company doesn't notice (and in the end what does it matter anyway)

I felt exactly the same when I read this. Plenty of people who are very good engineers have nothing in public to show.

Entire post can be summed up as "this is my answer, here is me rationalizing correlation=causation without any empirical evidence".

We could dissect the entire thing but really, if that is how it works for you, fine. Just don't push it onto others as the absolute truth. If this stuff was really so great, empirical research would've hammered it home decades ago. But it doesn't, and continues to struggle finding any meaningful correlations between these "whacky fun strategies" and actual job performance.

This isn't used widely because it is hard. But it's not but novel.

1. Companies want to standardize their hiring when really they should be looking to customize it to each candidate.

2. Lots of companies want to spread the blame of a bad hire across a committee of 4 or 5 people, but I think if you looked you'll find many startups doing it this way. They stop when they grow to a large size. They do it because frankly hiring good engineers is no longer the top priority (it's typically quotas AKA butts in seats).

> "My evidence isn't that great I admit. I've only helped build 3 unicorns (helped found one of them) and started 3 $1M+ companies as a solo founder."

Again with the awkward appeal to authority.

Cut straight to it. Your post's title is "how to hire actually good engineers". So how many of those have you actually hired? How many of them stayed there 2 years into their roles? How do you know you've actually hired "good" engineers, and not just people you yourself thought were good enough at the time you interviewed and hired them? Have you actually followed up on your hires 3/6/12 months into their roles, to check that they're still as good as you thought they were?

By the way - "$1M+" sounds like a particularly low budget for "actually good engineers", ironically enough.

false attribution error and appeal to authority

which was on top of a false dilemma to begin with

amazing

I get that the logical move is to get defensive, but just consider introspection

“What if they have no code they can legally show”

> Sorry to be blunt, but chances are they aren't very good, and they are definitely not as good as they think they are.

and there it is, one of the other subthreads is making fun of this kind of employer that swear by showing code

this is a false attribution bias where you decided not to notice that every evaluation technique the entire industry figured out will mostly by filled with people that cant code

your technique is simple no better

> every evaluation technique the entire industry figured out will mostly by filled with people that cant code

Is there a typo? The sentence doesn't make sense to me

Every evaluation technique will be mostly filled by people that cant code

The industry tries them all

Its the wrong conclusion on the efficacy of the technique

Hmm, and evaluation technique cannot be filled by something

One cannot fill an abstract concept like a technique with things

Maybe you mean that every evaluation technique will detect that most of the job applicants aren't good at coding? And therefore, all evaluation techniques seem to work? (Although some techniques are a lot better than others)

If so, then, yes I agree. That's a good point

Yeah thats my point, but also that none are better than the other

They are poor methods of screening or evaluating good candidates for the actual job duties

Ok then I understand :-)

Very useful!

Posted this on behalf of the silent majority that likes the approach.

Yeah we do this but also give people an option of either a take home test or live challenge. It works really well and fits all developer types.

At the end of the day all your after is signal on how well they understand software. There are lots of ways of doing that

I am out.

My good code are paid for and thus is owned by someone who is not me and can't be shared with a third party. The code that own is inherently bad as I want to create things as fast as possible without being bogged down by proper code writing etiquette.

On average, who's going to be better at writing code, the one who has done a side coding project in the past 10 years whilst developing, or the one who hasn't but only worked a job? (assuming everything else is equal) - The one who has more time for side projects might also have more time for work, too. Unfortunately, it does remove good people, since not everything else is equal.

IMHO It's still better than leet-coding questions because most developers don't need to use the skills gained from practicing leetcode when at work, whereas open source contributions and projects are valuable. Also, someone with no time outside work and family would struggle with both leetcode and having side projects.

> On average, who's going to be better at writing code, the one who has done a side coding project in the past 10 years whilst developing, or the one who hasn't but only worked a job?

You’re clearly leading toward the former, but I could just as easily make an argument for the latter.

In my experience, people with edifying jobs (usually because the problems are harder), are less likely to need to scratch the same itch outside of work.

I think requiring side projects not only shrinks the candidate pool, but also leads to adverse selection.

I've hired about 25 developers so far. So far, the 8 developers out of those 25 who had actual side projects (even not production ready and with caveats they told me ahead of time) have ended up being head and shoulders above the others.

Of course, there's not enough data to make a foolproof conclusion but I'd say that so far for me, having side projects that a dev can show is a clear indicator that they are an interesting candidate.

And when counting side projects, I count tools developers make to make their life easier and in my experience even great developers with edifying jobs will have situations where they create some small tools to help make their life easier.

But time is the important part here - a single parent raising two kids is not going to have time to code at home versus a twenty-something year old with no responsibilities and dependants who has an abundance of free time. Selecting in that manner can also end up being a discrimination (those from poorer backgrounds are more likely to need to care for relatives, same goes for women, who on average do ten hours more of unpaid care than men a week)

I just can't understand the worldview differences of some interviewers.

Everyone has a finite amount of time, they have other shit to do. The vast majority people irl just code for work. Are they bad programmers? Of course not.

I would say for the average employer paying an average salary, demanding you to showcase a portfolio of hobby project is plain obnoxious. The interviewer can hope the interviewee does some hobby projects and lives and breathes coding but demanding it as a baseline assessment for an interview is absurd.

For the average programmer when they have absolutely have nothing else to do maybe they will work on their sideprojects which even takes a backseat because they have to keep learning new crap every other week to make sure they are up to date.

> The one who has more time for side projects might also have more time for work, too

So you’re expecting people to work more than 40 hours a week?

You can't really make that comparison between individual developers though as there is just too much variability between jobs and companies. Some people are able to fully invest their creative capacity at their job and there are others that don't because either their job doesn't allow it or they have interests that are distinct from what the job requires.

Everyone in this thread seems to be hung up on sharing code from an employer or 'good code'.

You could clone an open source repo and use that if you're worried. The value isn't the code; it's the conversation.

"Bring your own code" is just meant to put the interviewee at ease because they are already prepared to talk about their own code.

> You could clone an open source repo and use that if you're worried. The value isn't the code; it's the conversation.

Open Source code plagiarism is a major problem and doing random forks/clone of popular repos to up your github cred is a thing and in my opinion it is unethical. Even if the interviewee tells you it is an open source project you will have a hard time distinguishing their code and the code from the original repo.

I think the interviewer should show their own production code and ask the interviewee what they think and what suggestion they have to offer. If you are not comfortable showing your own production code ask the interviewee to explore an open source codebase.

The "Show me what you got" approach puts way too much unjust pressure on a candidate and might force them to chose unethical means just to impress you. Then again if you want be impressed and if it is working so far for you, ignore my whole argument.

Who said th ex interviewee must present 'own' code there?

The conversation can just as well be around a library, product or system that isn't authored or contributed to, by the interviewee.

'What architecture is used here, and what are the down and upsides in this implementation'. 'What would you do different'. 'Which part do you admire, and what don't you like'. Etc.

Don't you think it is implied?

Also why would I voluntarily talk about codebases authored by anyone other than me while having in-depth knowledge about it? Nobody looks under the hood of things that interests them, let alone be critical of it’s design.

Talking about product and system design is something that is vastly different than stand-alone codebase. If you as an interviewer are interested about something you have the capacity of directing the conversation flow toward that. Ask them introductory questions about a product/system, gauge their interest in it then ask their opinion about it.

> Nobody looks under the hood of things that interests them, let alone be critical of it’s design.

Do you honestly believe this?

Many people do this. I do this. It's the best way to learn¹. There are repeating Ask-HN threads requesting for "high quality Open Source Software to learn from" for example.

¹Edit: on second thought: probably not the "best" way. Not for me, anyway. Just a good way.

Folks were responding to "Bring code you've written" which is different from just bringing some code.

> Everyone in this thread seems to be hung up on sharing code from an employer

Probably because you said “Bring code you've written”

I'd still show them my bad code, since I do pretty much the same thing. It would be a good opportunity to show how I'd refactor the side project if I could dedicate more serious time to it.

This is definitely a concern. I've previously worked for e.g. eye bee... and despite having written massive amounts of code in c, java etc, I'm probably violating some major NDA if I show any of that code, or discuss details of a first-in-the-world active-active HA cluster solution in the 90s.

By comparison, my github or personal code would be just a bunch of perl, rexx, sh/bash and python scripts written just to automate some work. I doubt anybody would think those should be cleanly written, highly maintainable, modularised, etc...

" just a bunch of perl, rexx, sh/bash and python scripts written just to automate some work. I doubt anybody would think those should be cleanly written"

I do. Even though my quick and dirty scripts are not always clean either, I still profit from the times when I take some more minutes to document what a certain script does and clean out at least the rough edges.

Otherwise it means rewriting the same thing again and again, which costs more time in the end. Or it means executing something you barely understand yourself anymore, which can be dangerous.

A lot of developers' best work is within employers' proprietary code bases, do you consider it a red flag when they share some of that code with you?

I have tons of open source code on GitHub to scratch my own itch, but nothing that I consider production ready.

It hasn't to be the 'best' work. The actual value comes from the conversation.

I've never been put in the situation where an engineer shared something alarming from a proprietary code base, but hypothetically if someone brought with them the code for how to hack superuser privs for an old company, that would be an ethics violation and I would pass.

Well it does not matter if it's alarming or not. They are in violation of their signed NDAs with their previous employer.

Also, if they don't work there anymore, they should've deleted all the code (if they had it) and surrendered/cleared up all the laptops.

But I like your idea, anyway I give big kudos to candidates with their own pet projects.

Almost every employer will violate their employee's rights without a second thought in ways they consider "unimportant but necessary to the business" ... as long as that continues it should be a 2-way street.

The point is not necessarily whether it's morally wrong to violate a contractual agreement when the other party is likely to do so.

It's that the interview process proposed by the G[...]P may force the candidate to do things that may be illegal to pass the interview. The candidate is of course responsible for their own choices, but the point is that as an interviewer, if you force your candidates to do this, you might short list those who have a tendency to violate contract terms.

This might work out for some, but I'd say it's a fair point to bring up.

> you might short list those who have a tendency to violate contract terms.

An asshole filter https://siderea.dreamwidth.org/1209794.html (maybe you've read already?)

I'm familiar with the general concept but never read that specific article. Fun reading though, thanks!

Mmm, no?

I'm an employer right now in my small company. Don't see any reason to screw up my employees, on the contrary, I see it's much better to work in a friendly environment. There's no such "unimportant but necessary to the business" things.

"Almost every employer will violate their employee's rights without a second thought"

What's your source?

"unimportant but necessary to the business"

What examples?

How are they sharing their code? At my company things are locked down so tightly that I can’t move code off of company computers. I would have to look at the code on one computer and type it into a computer of my own. Unless people are doing interviews on their company computer or the company is careless with their code, I’m not sure how someone would even share their work code.

Not sure how locked your computer is but everywhere I've ever worked, it's been trivial. Email the source file, zip it and upload somewhere, pastebin temporary, airdrop it, usb sticks.

My client is in the financial industry and have to use their equipment as a remote contractor.

The win10 laptop is locked down tight, including removable drives disabled, DNS forced through corporate servers, SSH blocked outside network, etc. I wouldn't be surprised if all activity was somehow centrally logged for compliance too.

I had to request permission to whitelist my VPN account to access Github.com. Even with VPN disabled the laptop still uses corporate DNS.

The security policy is designed to prevent theft.

Of course there _are_ ways to circumvent these protections but you'd be in a world of legal trouble if caught.

Working in the financial industry as well, not as a contractor but within the organisation itself.

To add to the above (everything is quite the same in my case): pasting medium chunks of data to external websites is restricted on the system level, same for uploading documents/images; emails to non-corporate addresses are scanned.

You would do that? Now I'm beginning to consider if I should be asking candidates to bring some intellectual property owned by their current/previous employer as a filter.

I am not saying I would do that (I wouldn't). I am just not sure how a computer can be locked down so tightly that it is impossible to send text from it to another computer.

Posted a comment in the above tread. To add to that everything one sends to print is being monitored. That leaves a room for leakage but it also makes the leakage tracing relatively easy, unless you have stolen other employee's ID (which is required to print), but do we really want to go that far for a job interview?

There are, of course, ways, but some organisations make the overhead way too big to bother.

I personally know of a case where the 'core team' of a major payments processing system was running development being completely isolated from the internet due to security reasons. Need to check stackoverflow - sure - have a walk to the cabin where you have a dedicated machine connected to web for that. Oops, and no copy-paste, sorry.

I could just print out and bring a patent.

All of those things you mentioned? Blocked. As an example, if I even plug any sort of device like a usb stick or an external drive to my machine, the computer locks up and security is on the phone with my manager within seconds.

Interesting, seems like my companies have been pretty relaxed then.

It should be. As should “bring your own code”. In which other engineering industry can you ask this of people?

We do the same thing and I find it’s a reasonable approach. For the candidate, it puts them at ease; they’re talking about something they’re familiar with (we go a step further and ask them to make changes to it).

Art, architecture, writing, music, I mean pretty much every creative industry.

Hence why I said engineering industry. Would you ask a chemical engineer to bring a recipe for a proprietary drug synthesis? The creative industry doesn’t exactly aim to obfuscate its methods.

Even better, would you ask a chemical engineer for their spare time projects? How would that even come about?

I graduated and worked for a few years as an electrical engineer. I had/have a portfolio of sorts and would gladly bring one to a job interview, including previous projects.

> Would you ask a chemical engineer to bring a recipe for a proprietary drug synthesis

Nobody is asking for anyone to bring complete codebases, either. Not to mention that, in chemical engineering, if something is proprietary enough, there are patents. Those come with credits.

I get the feeling that programming is the only industry paranoid enough to treat the most utterly mundane work as if it was bomb codes, averse to recognition enough to not credit people and disorganised/rushed enough so that 99% of companies are unable to let workers share their findings with others in the field.

Very few of those jobs have output that is obfuscated from public view.

Sounds good maybe for a founder or early in a startup, but otherwise this would probably illustrate the mismatch in priorities that exists between a single person's passion project, and risk-averse enterprise groupthink.

I think it’s really important to differentiate the code and the coder. Where we use this technique, I actually say to candidates, “I don’t care about the language / quality / age / immaturity of the code- it’s just something that you should be familiar with”. If someone turned up with an open source codebase to talk about it would be absolutely fine too.

Whenever I see a comment that starts out bragging about someone’s accomplishments I always assume the rest is going to be simplistic at best and nonsensical at worst.

I was not wrong this time either.

I recently interviewed at Whereby and they did exactly this. It was a breath of fresh air and I really enjoyed it.

If you didn't have an existing code, they suggested you built something from scratch and compensated you for the time spent.

Sure - I'll be happy to show you the scores of scripts I have written to do every tasks like switch pipewire audio sinks, wrapper to youtube-dl, generate/copy rsa token, convert/combine images to pdf etc that were written in the sliver of time afforded to me after I'm done family and kids. Those scripts in no way represent the code I get paid to write in my job but if you find that to be an an issue, then you have found the flaw in your method.

I think this is a great interview. I do something similar.

I’ve done most of my interviewing in finance where:

1. The engineers are mostly really good

2. They almost never have side projects, and everything they have ever done is super secret and proprietary.

For this it makes sense to ask people to do a take-home, and then ask them questions about it when they come in.

I usually also ask them to add a simple feature to their code and just silently watch them code it. You learn a lot from watching how someone breaks down a simple problem, especially where they are expert (ie they are working in their own codebase).

Sounds significantly worse then whiteboarding.