News for hackers. Stuff that matters.
Re:How to Hire Great Engineers by protonimitate on Thursday July 30, 2020 @01:44PM (#24000971)
I really like the idea of "chopping the onion". I've personally had a handful of devs who passed our interview process with flying colors, but couldn't actually deliver anything once they got in the door. They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.

> The grading is automatic. I think is ok as _one_ metric, but I wouldn't want it to the be the _only_ metric that is being looked at. ICs aren't really ICs. You can be a JIRA champion, knocking tickets into the done column at the speed of light, but Great Engineers do more than just that.

Great Engineers build up their team, and help others grow. They know how to push back on unrealistic asks. They can incite help when they get stuck. They can translate complex issues into digestible bites. Writing good code is the bare minimum quality of being a Great Engineer.

The main barrier I see to hiring Great Engineers is that you really need to hire Great People. But a lot of hiring practices are taking all the human aspects out of hiring (except for the "behavioral" questions, but even these can be gamed like system design or leetcode). Great People worthy of hiring are driven, passionate (not "spent all my waking time coding", but "truly believe in the mission/goals of the company/project"), empathetic, intelligent, and trust worthy. None of those traits are easily discernible from random logic puzzles.

I don't know what the solution is. At some point there has to be a "can actually write code" litmus test, but I don't believe any of the current methods work well to prove that. I think take home tests are closer to the "chopping onions" idea, but only require commitment from the applicant, not the employer, which leads to a lot of wasted time.

Personally, I think temp-to-hire or work-for-a-day type models could work, if there was some standards to prevent abuse. I would rather sacrifice a few days to a week to see if I actually like working somewhere, and to prove to them that I can do the job, then look at CTCI ever again.

Re:How to Hire Great Engineers by bonoboTP on Thursday July 30, 2020 @02:26PM (#24001394)
> "truly believe in the mission/goals of the company/project"

Well, first make sure there actually is some "mission" there to "believe in". In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money. Requiring passion is a euphemism for being over-workable. That you'll work outside working hours, you can be called during vacation etc, because of course this job is the mission of your life, right? Otherwise, you're not passionate...

In practice, these "passionate" people are often young and naive or just really ambitious and work their ass off to build their resume and will be on their way soon enough.

A good engineer contributes high value and knows this. And therefore will make the company pay up, and will not take any bullshit. And being that good, they will be picking where to work and you as the company compete to get them. In these cases the interview is about the engineer judging the company and looking for exactly things like the requirement to "believe in the mission" or whether the conversation is on a professional, adult level.

Re:How to Hire Great Engineers by Godel_unicode on Thursday July 30, 2020 @07:18PM (#24003854)
> Requiring passion is a euphemism for being over-workable.

That's definitely true in the general case. Having passion means wanting to do the best overall thing because you care about the thing going well. The best overall thing is working hard to prevent long hours which lead to burnouts. If you have managers and senior leaders who share the engineers passion, this problem is self-correcting.

Source: I frequently lecture our (junior esp) new hires about working too many hours. Especially during incidents. Yes this is exciting. Now go sleep and see your family. Come back when you are rested and ready to play at full speed.

I also have uncomfortable talks with people who frequently say "meh, close enough" about whether they actually want to be here. My team performs at a consistently high level.

Re:How to Hire Great Engineers by rhizome on Thursday July 30, 2020 @06:00PM (#24003325)
Not to mention that most companies don't need great engineers. Average engineers for average companies is perfectly fine. Of course, then the company leadership has to be realistic about their place in the universe, but that's a separate problem, plus hiring authorities can have a realistic view of the big picture and sell their hiring selections as "great engineers for a great company" to management.

It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.

Re:How to Hire Great Engineers by bonoboTP on Thursday July 30, 2020 @06:56PM (#24003725)
> It's kind of mutinous, but it's not like there's anything to be gained from loyalty to management who aren't loyal to employees.

It goes both ways. The current equilibrium, at least in the US, is that (SW/IT) engineers leave after ~2 years, and it's causing quite a bit of friction. But it's hard to move out of the equilibrium.

Re:How to Hire Great Engineers by rhizome on Saturday August 01, 2020 @11:48PM (#24026170)
>But it's hard to move out of the equilibrium.

I don't think it's so much an equilibrium so much as a battle, denoted by "friction," but I'm sure it would come down to each saying the other one defected first. I think we can figure out where the disloyalty started, calculated from the first principles of capitalism.

Re:How to Hire Great Engineers by bonoboTP on Tuesday August 04, 2020 @07:50AM (#24048991)
By friction I meant all the onboarding costs and time, and the lost knowledge evaporating with the leaving employees. There's tons of waste there.
Re:How to Hire Great Engineers by protonimitate on Thursday July 30, 2020 @03:38PM (#24002085)
> In 99.9% of the cases people work on perhaps technically exciting but impactwise boring industry projects, whose main mission is to make money.

Passion, imo, is an attitude - and is reflected in quality of work and teamwork. I do think it is often used to trick people into getting less than they deserve, but doesn't have to be the case.

When I say I want to work with passionate people, I don't mean people who are willing to work 80 hours a week for 10 dollars an hour because they love coding or want to solve world hunger. I mean I want to work with people who give their best, put the user/experience first, and are genuinely motivated to do good work. This doesn't mean you have to fake smile, or get taken advantage of, it just means you have bring some positivity to the table.

Re:How to Hire Great Engineers by vsareto on Thursday July 30, 2020 @05:47PM (#24003219)
Some people give their best, but sometimes it isn't good enough. In those cases, the company will just look for someone else. Then that passion counts for little.

This is the issue with signalling passion - it causes people to drive their expectations of you higher. You have a higher chance of not living up to someone's standard.

Re:How to Hire Great Engineers by Animats on Thursday July 30, 2020 @08:40PM (#24004460)
Requiring passion is a euphemism for being over-workable.

Yes. Makes me think of Zappos, the shoe store. They're big on demanding employee enthusiasm.[1] But in the end, they're just a shoe store.

[1] https://www.zapposinsights.com/about/core-values

Re:How to Hire Great Engineers by carlosdp on Thursday July 30, 2020 @06:46PM (#24003661)
> Personally, I think temp-to-hire or work-for-a-day type models could work

I've seen the contract-to-hire stuff come back up recently. There are a couple big flaws with that model, beyond possible abuse, though.

1) You basically are excluding yourself from any candidate that still has a full time job. It would be crazy for them to leave their current gig for a place that might not actually bring them on full time.

2) It doesn't scale, bringing on new team members properly takes time and effort. Not only are you potentially wasting their time, you are also wasting your team's time, and you can only do this kind of evaluation so many times.

I do agree though that the standard interview model doesn't make sense. Memorizing leetcode problems doesn't map to good engineering, and take home problems filter out a lot of good engineers that won't bother spending the extra time.

Re:How to Hire Great Engineers by ilyaa on Friday July 31, 2020 @12:31AM (#24005678)
Hey, thanks for insightful feedback (blog post author here)!

I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.

Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)

We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.

Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.

Re:How to Hire Great Engineers by collyw on Friday July 31, 2020 @04:51AM (#24006908)
> They could recite any leetcode problem from memory,

There is your problem. You are testing for something unrelated to the job.

Re:How to Hire Great Engineers by Fiveplus on Thursday July 30, 2020 @02:10PM (#24001259)
> handful of devs who passed our interview process with flying colors, but couldn't actually deliver anything once they got in the door. They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.

Interesting, why do you think was this the case?

Re:How to Hire Great Engineers by whichquestion on Thursday July 30, 2020 @02:30PM (#24001424)
Coding is not the only thing required to be an excellent developer and contributor to your team and company. Sometimes you must gather requirements from users / customers. Sometimes you have to work with other teams or products to integrate them into your product.

There’s also the aspect of working with the rest of your team on whatever problems you are solving and not siloing yourself to just the problems you’re working on.

That and when you’re hit with new and interesting problems that have never been solved before, you cannot look those up in the leetcode manual and memorize them to solve. You have to actually create something on your own and apply whatever you’ve learned from programming in the past.

It’s like playing a survival video game with the detailed how-to always at your fingertips versus being in the wilderness having to survive on your own wits without a guide.

Re:How to Hire Great Engineers by walshemj on Thursday July 30, 2020 @06:10PM (#24003417)
It does seem to be a thing with younger developers, who want to stay wfh and just work on jira tickets.

Any sensible developer learns early on that mostly the coding is the easy stuff - what Is the problem I am solving here?

Re:How to Hire Great Engineers by gedy on Thursday July 30, 2020 @02:22PM (#24001355)
I see this time and again from observation over 20 years. Also many great engineers don't have cs backgrounds and don't do well on whiteboard/leetcode
Re:How to Hire Great Engineers by alpha_squared on Thursday July 30, 2020 @02:17PM (#24001315)
Not the person you're responding to, but my perspective is that there is often a false equivalency between academic understanding (solving hard/clever problems) and productive development (meeting deadlines, building/fixing projects). Many of the stereotypical developer interviews assume "good developers" who are good at one are good at the other. My anecdotal experience is that most are not, they excel in one or the other.
Re:How to Hire Great Engineers by Olreich on Thursday July 30, 2020 @02:28PM (#24001403)
I don’t think they are exclusive, you can be a great engineer that knows all about the algorithms in CS books and keeps up on the latest research in parts of the academic field. They aren’t even different skill sets, just different places to find the knowledge. Game development and cryptography bear this out well. The top engineers are the ones that read all the papers and that can build solid systems based on them.
Re:How to Hire Great Engineers by alpha_squared on Thursday July 30, 2020 @02:51PM (#24001635)
I don't think I framed it as exclusive, but that the intersection of both sets of skills is uncommon (I attest they are two different sets). Sure, you can aim to hire only people who excel in both, but the typical software interview certainly doesn't assess the latter.
Re:How to Hire Great Engineers by jmchuster on Thursday July 30, 2020 @07:51PM (#24004127)
some common engineering things that leetcode doesn't test for

    - using http apis and transforming the resulting data
    - using database queries and transforming the resulting data
    - navigating a large codebase
    - figuring out what tasks are important to accomplish
    - researching things you don't know
    - debugging a complex issue
    - evaluating tradeoffs
    - general engineering intuition
Re:How to Hire Great Engineers by richajak on Saturday August 01, 2020 @11:18AM (#24021093)
Doer/implementation vs abstract/big idea thinker.

thanks, can be useful tips for my future staff recruitment.

Re:How to Hire Great Engineers by PopeDotNinja on Thursday July 30, 2020 @02:23PM (#24001363)
It's like memorizing the answers to an IQ test. A relative of mind knows all the answers, as he conducted the test on many occasions, but he doesn't have the max IQ.
Re:How to Hire Great Engineers by rightbyte on Thursday July 30, 2020 @06:12PM (#24003426)
Ye. Making up quick sort sort on the fly without ever seeing the solution would be really impressive, but reciting it from memory doesn't say much. If someone would implement "max sort" on a interview at-least you would know he didn't practice for the "IQ test".
Re:How to Hire Great Engineers by kamaal on Friday July 31, 2020 @06:30AM (#24007322)
>>They could recite any leetcode problem from memory, but didn't know how to contribute value to their team or company.

That's understandable any one with that much proficiency on leetcode would be wasting their whole day on that site than building real things.