News for hackers. Stuff that matters.
Re:Senior devs. Is anyone else insulted by coding exams? by commandlinefan on Wednesday March 09, 2022 @11:20AM (#30616234)
If those tests actually filtered out incompetent programmers, I'd say they were great, but my experience is that they all test very shallow knowledge that doesn't relate to ability to produce software that performs under load.

If you could come up with such a test, it ought to just be used as a standard for everybody to take.

And yes, as unpopular an opinion as it is around here, I'd love to see a "bar exam"/licensing board for software developers.

Re:Senior devs. Is anyone else insulted by coding exams? by lijogdfljk on Wednesday March 09, 2022 @11:39AM (#30616531)
I've had moderate success discussing Rust (a language we hire for) at moderately low levels.

Ie, i've had the thought that it's difficult to bullshit topics with any depth. Even basic questions, why isn't an Rc safe to send to a thread? How does an Rc work? What is the difference between an Rc and an Arc? etc

I'm happy to accept "i don't know", those are valuable too. But the whole conversation process led me to believe a lot about the candidate. We've hired 10+ juniors and seniors with this process and i've not felt cheated yet.

I still feel like i need to find a way to open up people who "freeze". I do that myself, so i can relate, and my strategy of interviewing is pretty 1:1. Still though, i've had more faith in it than live/take-home tests.

Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @12:57PM (#30617805)
> I'd love to see a "bar exam"/licensing board for software developers.

Can't resolve IRQ conflicts? Don't know the big-O complexity of different Delaunay-triangulation algorithms? Out of date on which features Microsoft added to LINQ in different versions of C#? Don't know where the "launch debugger" button is in Android Studio? Sorry, we aren't allowed to hire you to work on Instagram's iPhone app.

Better hope the police don't find out your Android has an unlocked bootloader; I hear prosecutors are really cracking down on unlicensed software development ever since the assassination. They say unregulated software development leaked his location to the Zetas, I don't know if I believe that.

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @01:02PM (#30617872)
Witness the world of unlicensed renegade civil engineers roving across the American landscape, building unauthorized bridges and infrastructure.

Or perhaps Robert De Niro's character in Brazil.

Re:Senior devs. Is anyone else insulted by coding exams? by pan69 on Wednesday March 09, 2022 @04:55PM (#30620518)
And I think this is where the crux of it is; not all software is equal.

Building bridges is very different from putting a shed in someones backyard.

Software is a spectrum and if your company builds sheds in peoples backyards then you don't have to hire in the same way you'd hire a civil engineer.

Lots of companies copy the Google style interview without actually being anything like Google or any understanding why Google hires that way.

Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @06:26PM (#30621343)
Man, can you imagine how great our physical infrastructure would be if the Linus Torvaldses and Fabrice Bellards of the civil-engineering world could actually do this?

Now imagine it was illegal for Linus Torvalds and Fabrice Bellard to do what they've done.

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @06:29PM (#30621375)
Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @06:57PM (#30621589)
It's difficult to interpret your comment. It's a link to a news story about how a partly built "illegal" stone bridge collapsed, killing 11 people, presumably because it was poorly designed. In context, you seem to be suggesting that Linux, QEMU, and ffmpeg are analogous to that bridge because they are of extremely poor quality, while there's some kind of official government software licensing body that protects people from such terrible software. Is that what you meant?

No such body exists, though many attempts have been made to establish one, with dismayingly destructive results, as I outline in https://news.ycombinator.com/item?id=30621151.

I think that if we want to enable the Fabrice Bellard of civil engineering to build bridges, we probably need to keep all humans at a safe distance from the bridge until it's fully complete, then stress-test it to validate the FEM model before opening it to traffic. Also, probably not build it out of brittle materials like stone, because the stress test wouldn't give you enough information to validate the model's failure predictions.

We should be thinking about how to make civil engineering more like software engineering, with rigorous version control, automated testing, automated builds, rapid improvement in the state of the art, and safety features that minimize risk to humans in case of failure. Not vice versa.

(Though it would be nice if more programmers were in the habit of planning ahead and thinking about failure modes.)

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @07:17PM (#30621727)
It's an unserious comment responding to an unserious discussion. You've provided a bevy of historical knowledge, which while interesting, in no way resembles what software engineering licensing would be like in the present day. For one thing, corporate power is so entrenched and regulatory bodies so enervated in this space (as everywhere), there is absolutely no possible way the licensure dystopia you've so vividly conjured could exist. And nobody on this forum calling for licensing is asking for some onerous top-down governmental bureaucratic licensing regime, so you've basically created a neat '90s cyberpunk retrofuturistic setting that has little to no relevance to the discussion at hand.

If anything, people are calling for industry standards, something akin to craftsmen guilds of olde, overseen by an existing professional association of engineers such as the ACM or the IEEE [0]. Alternatively, perhaps it could arise as the end result of Triplebyte or some other private solution trying to be the Collegeboard of this industry. Speaking of which, academic exams such as the SAT, the ACT, or AP tests have no legal binding, but most higher academic institutions (until recently) choose to use them as standards. So it's more of a widely-subscribed informal standard.

In practice, such a license wouldn't hold too much water anyway, certainly FAANG would ignore it for the most part in favor of their higher standards, but perhaps it could at least be a Leetcode replacement for smaller firms [1]. That would save both applicants time in having to run the LC gauntlet over and over again and every single company they interview with, and help those smaller companies avoid their hiring woes of cargo-culting FAANG interview practices and accidentally filtering out every applicant along the way. None of this would forbid the practice of programming by non-accredited practitioners, as the licensing regime bogeyman you fear would have absolutely no teeth for enforcement.

Finally, if formalizing the engineering nature of software engineers means that American engineers can be like their Canadian counterparts and wear iron rings [2] to mark their achievement and profession, then that is infinitely more cool than the status quo and more compelling than any number of fantasy dystopias, or evocations of the state of the art of '70s computing, or Operation Sundevil, or the goings-ons in... Iran.

[0] https://news.ycombinator.com/item?id=30616114

[1] https://news.ycombinator.com/item?id=30493041

[2] https://en.wikipedia.org/wiki/Iron_Ring

> We should be thinking about how to make civil engineering more like software engineering, with rigorous version control, automated testing, automated builds, rapid improvement in the state of the art, and safety features that minimize risk to humans in case of failure. Not vice versa.

Have you seen the state of software engineering? I'm not sure if this field is in any state to be acting smarter-than-thou towards other engineering disciplines.

Re:Senior devs. Is anyone else insulted by coding exams? by zozbot234 on Thursday March 10, 2022 @04:45AM (#30625078)
> If anything, people are calling for industry standards

Industry certifications do exist. They just haven't been very successful so far. Perhaps we're going to see more focus on them if the pace of development and disruption slows down, but as it is I don't think there's much interest in them.

Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @07:48PM (#30621945)
Not only could it save us time on job interviews, it could save us time on tech support calls: https://xkcd.com/806/

I think you're being unrealistically conservative about how much things are going to change. You might be right that things will get better instead of worse, but overall I think you're seriously underestimating just how much and how fast the world changes over time. Don't forget that it's only been 21 years since "Know Your Customer" was an unthinkable Big-Brother sort of idea, only a year since "vaccine passports" were widely thought to be a paranoid fantasy that could only happen in Israel, and only two weeks since Russia invading Ukraine was widely dismissed as implausible.

More broadly, I think that if you want to know what's going to happen 50 years in the future, you need to look at things that have happened at least 50 years into the past. 02072 will probably be as strange to us, if we live to see it, as our life today would have seemed in 01972.

I'm not sure why you're dismissing Iran. 85 million people live there and their welfare is important.

As for "smarter-than-thou", I think it's rather that good programmers are acutely aware of the severe limits on human mental capacity and therefore look for solutions that don't require them to be infallible or even smart. Also, we have computers, so we can build our practices around them!

A couple of years ago, Hillel Wayne interviewed a number of "crossover" engineers who had worked both as software engineers and in other engineering fields, and my recommendations above are strongly influenced by what they said about what the different fields can learn from one another. If you're interested in these issues, you might be interested in reading his writeup: https://www.hillelwayne.com/post/what-we-can-learn/

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @08:06PM (#30622060)
Actually, if you had wanted to debunk my proposal, this classic would've be the one to do it: https://xkcd.com/927/

> I think you're being unrealistically conservative about how much things are going to change.

They will change, but the direction you are invoking is fundamentally in the opposite course of history wrt this industry. When has there been greater government control over the software industry, specifically when it comes to what constitutes a software engineer?

> You might be right that things will get better instead of worse

I said corporate control is rampant and regulators are as weak as ever. I supposed from a certain political point of view things are getting better.

> only two weeks since Russia invading Ukraine was widely dismissed as implausible

The Orange Revolution happened in 2005. I'm sure technothriller novelists and wargamers have at least been considering the current scenario since then.

> I'm not sure why you're dismissing Iran. 85 million people live there and their welfare is important.

Because this entire discussion has been mostly about Silicon Valley-adjacent engineering. Perhaps Leetcode interviews are a pain point for Iranian programmers. But if you're bringing up government tyranny in civil rights in one area to dismiss a licensure scheme - or more accurately, certification [0] is just plain ol' motte-and-bailey.

[0] https://news.ycombinator.com/item?id=30493814

> Also, we have computers, so we can build our practices around them!

Computers are only as good as those who write the code, unless we are to start building our practices around the results of neural networks, which is a most intriguing proposal.

That Hillel Wayne article is a good read, but again, I am calling for humility. Because much of the engineering in this industry is far from the lofty bridges and oil rigs the word implies: https://www.karllhughes.com/posts/plumbing

And to return to Wayne, consider part one of his series: https://www.hillelwayne.com/post/are-we-really-engineers/

Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @08:26PM (#30622195)
Why would I want to debunk your proposal? I don't think it's bunk, I just think it would have some undesirable consequences. That's no bar to pointing out the desirable ones!

This is the first time Silicon Valley has been brought up in this discussion, but keep in mind that practices in the US are influential around the world, and the harms from strengthening inequality and substituting credentialism for meritocracy fall predominantly on people outside the US. The discussion has mentioned YC (Boston originally), Rust (Mozilla, San Francisco, previously SV), Microsoft (Washington), Android (SV), iPhone (SV), Instagram (SF), the Zetas (Mexico), Brazil (the movie), the American landscape (much of which is in Brazil, the country), Google (SV), FAANG (SV/Washington/SV/SV/SV, but all with offices worldwide), Linus Torvalds as a student (Finland), Fabrice Bellard (France), Guangdong (China), the ACM (US-wide), the IEEE (also), Triplebyte (SV but kind of worldwide), the SAT/ACT/AP tests (US-wide), Ukraine, Russia, and Canada.

I agree that the danger of war in Ukraine was foreseeable; I wrote about it in 02016 https://dercuano.github.io/notes/wwiii-genesis.html https://dercuano.github.io/notes/state-of-the-world.html and 02017 https://dercuano.github.io/notes/2017-sap-allocation.html and George Kennan wrote about it last millennium. But it's also true that it was widely dismissed two weeks ago.

I'm glad we agree on calling for humility! And, yes, Wayne's whole series is worth reading.

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Thursday March 10, 2022 @02:31AM (#30624146)
On the same note of this prior conversation, what timing for this to be #1 on the front page:

https://news.ycombinator.com/item?id=30623617

Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Thursday March 10, 2022 @02:47AM (#30624264)
Heh!
Re:Senior devs. Is anyone else insulted by coding exams? by commandlinefan on Wednesday March 09, 2022 @01:01PM (#30617865)
Yep, that's exactly the sort of ridiculous hyperbole that gets thrown around every time somebody suggests any sort of standardization around professional software development!
Re:Senior devs. Is anyone else insulted by coding exams? by kragen on Wednesday March 09, 2022 @06:03PM (#30621151)
There's no reason to think it's hyperbolic or ridiculous.

We've seen lots of sorts of standardization around professional software development, with uniformly dismal results, and lots of criminalization of experimentation by astoundingly incompetent authorities.

— ⁂ —

The first official standard for professional software development was COBOL, shortly followed by JOVIAL.

In the 01970s the US military standardized on Ada for, among other things, their hard-real-time systems; it took about a decade of violating the standard to convert Ada into a system that was somewhat suitable for hard-real-time systems, after which an Ada-induced problem destroyed the first Ariane 5.

The ACM withdrew from the SWEBOK process after it became apparent that knowledge of actual programming was going to be completely excluded; instead SWEBOK certifies that you endorse the obsolete, counterproductive management processes to which "agile development" was conceived as an antidote. (If you're familiar with the BOK for some other branch of engineering, you may be aware that, for example, the structural engineering BOK includes not only project engineering practices but also what makes structures stand up or fall down, and the civil engineering BOK includes things like engineering mechanics, strength of materials, and chemistry. Presumably this is because political decisionmakers have been able to observe the difference between successful and unsuccessful skyscrapers and dams and infer which engineers were responsible.)

The A+ certification for IT work is impossible to get if you only know Linux and not Microsoft Windows, and last I saw, it did still require that you knew how to debug IRQ conflicts—long after ISA buses were limited to PC104 systems.

PCI-DSS audits routinely, though not universally, mandate counterproductive security practices like installing virus scanners on Linux servers.

If we'd effectively standardized professional software development in 01960 we'd still be programming in COBOL today.

Moreover, the Board of Licensure could revoke our licenses when we criticized widespread bad software development practices, as has happened to Charles Marohn of Strong Towns: https://www.strongtowns.org/journal/2021/5/23/lawsuit

— ⁂ —

You may not have commemorated it, but on March 1 we passed the 32nd anniversary of the Secret Service raiding Steve Jackson Games for publishing a "manual for computer crime", namely, the Cyberpunk module/game for the GURPS role-playing game/system.

Outside the US the situation is often even worse; Saeed Malekpour was sentenced to death in 02010 because he wrote open-source image-uploading software and a porn site used it without his knowledge. (His sentence was suspended; he escaped in 02019.)

Do you really think it would be a good idea for the governments that chose those prosecutors to decide who gets to program and how?

— ⁂ —

The bigger issue is that computers are now our exocortices; we've offloaded significant amounts of our thinking and communication onto them, as we did previously with inventions like books and clocks. Computers vastly extend our mental abilities. Without control over our computers—without the legal and practical ability to program them and to investigate what they are programmed to do—we in effect lose our freedom of speech, association, and thought. That power is far too dangerous to allow any guild or faction to monopolize it.

Re:Senior devs. Is anyone else insulted by coding exams? by juancn on Wednesday March 09, 2022 @11:27AM (#30616356)
I mean, you don't need a really hard test to weed out most hacks. In any case, if the test is hard, companies that know their shit in hiring, care more about how you approach it than what the end result is.

In the end, the interview tries to figure out how it's like to work with you. Do we get along? Can we communicate effectively? Will you be happy and thrive here?

Re:Senior devs. Is anyone else insulted by coding exams? by nouveaux on Wednesday March 09, 2022 @11:53PM (#30623357)
You sound like a great hiring manager/interviewer. However, I have read stories of people who got rejected only because they did not finish the coding challenge in time. The engineer reported that the candidate communicated well, but did not finish. Hiring manager rejects the candidate.
Re:Senior devs. Is anyone else insulted by coding exams? by juancn on Thursday March 10, 2022 @01:25PM (#30630485)
Yeah, there are bad employers and bad interviewers. Sometimes the importance of it is underplayed. I always drill others that interview duty is the most important thing you can do at a company (even though it sometimes feels like a chore). The people you help choose, influences culture and long term viability of the company. Take it seriously and be fair with candidates.

Having said that, figuring out a fair and effective interview process is HARD. It requires constant tweaking and monitoring. Also, you have to start from a decent set of values at the company.

For example, we do shadow interviews. When we certify interviewers, we have them come to watch and listen to an interview (we explain to the candidate beforehand what's going on), and then after the interview ends, we meet with the shadow interviewer and discuss what they picked up, and explain why I went such and such way, what I was looking for, why such and such mistake is not a big deal, and so on.

After a couple of these, we have them interview, but with a certified interviewer shadowing them. Just observing (we have them practice on us their interview exercises if any before the interview).

And the we give feedback, you rushed a bit here, give them more time to be at ease with you, did you notice such and such, whatever... If everything goes well, we certify them, and they can fly solo.

We try to have a varied pool of interviewers to avoid biases. The downside is that you need to do more interviews as a candidate, to get a fair assessment (i.e. no single person can derail you).

In the end, we've all been in a crappy interviews and try our best to not inflict that on other people.

Re:Senior devs. Is anyone else insulted by coding exams? by AnimalMuppet on Wednesday March 09, 2022 @11:55AM (#30616790)
> ... they all test very shallow knowledge that doesn't relate to ability to produce software that performs under load.

That's a later filter. The first filter is, can they code at all. Like, FizzBuzz level of "at all".

I mean, if you need them to produce code that performs under load, you'd better interview for that too. But people tend to first worry about whether they can clear the lowest bar before they proceed further.

Re:Senior devs. Is anyone else insulted by coding exams? by sokoloff on Wednesday March 09, 2022 @12:37PM (#30617536)
It is difficult to comprehend how low-skilled the shallow-end of the applicant pool is. I think it’s gotten worse, not better, since the original FizzBuzz essay was written.
Re:Senior devs. Is anyone else insulted by coding exams? by starlust2 on Wednesday March 09, 2022 @12:39PM (#30617567)
They actually do.

The test isn't (or shouldn't be) that you understand string manipulation (or whatever) but rather that you can work through a new problem. There are many programmers who claim many years of experience but have poor problem solving skills. They can implement a CRUD form and basic tasks, but fall flat when faced with a problem they've never encountered before.

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @11:38AM (#30616525)
The thing about the license is that it avoids the redundancy of having to pass the same orthogonal algorithm questions at every single company an applicant chooses to apply at. That alone is a great time-saver, and allows applicants the freedom to focus on actual work.
Re:Senior devs. Is anyone else insulted by coding exams? by jxidjhdhdhdhfhf on Wednesday March 09, 2022 @12:27PM (#30617327)
Not really. You'll still have interviews, they'll just be more personality and resume-driven.

Personally, I'll take writing FizzBuzz over and over again vs. "where do you see yourself in 5 years?" ("I don't know but probably not working here") and "why do you want to work for our company?" ("to earn money").

Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @12:31PM (#30617411)
Most interview loops include Leetcode for the technical screen, then at least one domain-specific coding section (like a more tense pair programming section) during the on-site, and maybe another algorithms section on the on-site (several if you’re Google). The license would at least eliminate the algorithms section to focus on the domain-specific, which is at least more reflective of real-world work.
Re:Senior devs. Is anyone else insulted by coding exams? by jxidjhdhdhdhfhf on Wednesday March 09, 2022 @12:42PM (#30617603)
Also, its a lot easier to study leetcode problems than it is to study for domain-specific ones (since there are many different domains). I would expect having more domain-specific problems would limit you to a narrower range of jobs.
Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @12:51PM (#30617718)
It all comes down to philosophy- one could argue that it's better to have interviews reflect the actual material of the work. And that such domain-specific questions would require less prep, as the candidate's experience should already be strengthening them to answer such questions. Not to mention additional career development in those skills- asking and answering Stack Overflow questions, technical blogging, side projects using the same stack, etc.
Re:Senior devs. Is anyone else insulted by coding exams? by jxidjhdhdhdhfhf on Wednesday March 09, 2022 @01:01PM (#30617866)
That's much worse than leetcode problems because it requires tons of work outside work. Most people, myself included, have a life and interests outside of the very narrow focus of their career, and don't want to spend 16 hours a day focused on that one aspect of life. Studying a few leetcode problems is no big deal in comparison.
Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @01:05PM (#30617908)
Depends on the companies you're applying to, it's rarely just "a few." One of the difficulties is retention. A LeetCode a day only goes so far until you have to repeat older problems. Better to get it all done in one test.
Re:Senior devs. Is anyone else insulted by coding exams? by jxidjhdhdhdhfhf on Wednesday March 09, 2022 @12:32PM (#30617451)
You're assuming they will actually eliminate it though rather than just pick something else to test.
Re:Senior devs. Is anyone else insulted by coding exams? by Apocryphon on Wednesday March 09, 2022 @12:34PM (#30617491)
Sure, but at least that might be something more relevant to day to day work, and require less grinding than Leetcode.
Re:Senior devs. Is anyone else insulted by coding exams? by commandlinefan on Wednesday March 09, 2022 @12:34PM (#30617483)
> where do you see yourself in 5 years

Mostly in mirrors. Unless I turn into a vampire.

Re:Senior devs. Is anyone else insulted by coding exams? by chrismcb on Thursday March 10, 2022 @12:45AM (#30623594)
I think you are looking at this the wrong way. the test isn't too find who can write software that performs under load. It is too find out who can write software, period. Essentially the fizzbuzz test. There are a ton of people who say they can write code, but then can't write a simple for loop.
Re:Senior devs. Is anyone else insulted by coding exams? by ryandrake on Wednesday March 09, 2022 @11:31AM (#30616422)
Oops, I posted my rant[1] about our need for a bar exam before realizing you got there first. Fully agree, this profession really needs this.

1: https://news.ycombinator.com/item?id=30616361