> If you're turning down someone who wrote space shuttle firmware, because they "can't do fizzbuzz", then you are administering a terrible test and getting a false negative.

Or... large companies, particularly defense, excel at hiding mediocrity in their ranks, and it's easy for someone who produces no or negative work to be handed around rather than fired.

Small companies also excel at hiding mediocrity.

What's the success rate for startups?

The reality here is that everyone is supposing their particular bias explains what happened.

No one seems to have asked what actually went wrong in the interview. Was the interviewee truly incompetent? Were they nervous? Were they fine at code but terrible at interviews? Was the language or environment unfamiliar? Had they just got off a plane after zero hours of sleep?

Without data, opinions are worthless. And generally, there's too little data about the relationship between interview performance and job performance, and too much unthinking mimicry: "We do this because everyone else does."

> Small companies also excel at hiding mediocrity.

> What's the success rate for startups?

While there are certainly mediocre coders in many many startups, let's put the blame for most startup failures firmly where it lies:

"It was a turd of an idea and no-one told the founder/the founder didn't believe it."

not

"It was an amazing revolutionary concept that was only sabotaged by dimwits who couldn't fizzbuzz."

Or not

That is why I advocate for a hypothetical "bar exam" for the software programming profession. I also posted a question concerning that: https://news.ycombinator.com/item?id=16702389

Going through a series of technical filters is not a problem in and of itself. The problem is that each company has its own set of filters, many times with skills that don't necessarily transfer to other interviews.

With the "bar exam" approach, you only need to pass 1 exam for N number of companies, leverage that to fast-track through a company interview. With the current approach, you need to take N exams for N companies, and return to square one if you fail. There is no extra layer of validation that lets you keep a "ahead" placement in these interviews. Company specific interviews should focus on basic soft skills, and if they mesh well with the team.

I'm an experienced programmer- not yet considered senior level, but have worked at several companies -and having a hell of a time going through my latest job hunt.

I have been applying for jobs for 3 years straight now, and it's telling that the only offers I've gotten were three short term contract gigs- one from a past employer who's already familiar with my work, and two from small studios with a very informal process where they just looked at my resume and Github projects and with that, they were very convinced that I'm the man for their job. I took these jobs mainly to add more skills and to fill up the gaping time hole in my resume.

But for the "regular" full-time jobs at businesses, big or small, local or on the west coast (I'm a midwesterner), I have not gotten an offer. I've been through many tests and live coding going through a lot of things. Triplebyte expects you to be a speed demon, but at least they give detailed feedback on what you did wrong.

So what is with the saturation of applicants and tendency to make the hiring process slower and slower? That's what these coding tests and interviews do, they make the process slower.

It's not 2009 anymore- unemployment is reasonably low, so this current phenomena of rigorous testing shouldn't happen. More programmers need to discover and embrace the power of collective bargaining.

I've been advocating for the same thing except I call it a "Software Engineering License".

The equivalent would be asking a math professor to do integration by hand. They can do it, but that's probably not the job they want anyway.

I'm intrigued by the on the spot performance aspect, so I wonder if anyone has performed a skit at an interview. It's a tempting idea. Some people are intrigued by live performance, especially the kind with constraints like this, people like Frank Abagnale for example.

Probably, most likely on the Ali G show. If you know enough inside details on the typical interview pattern at a company this would be easy to do, and you would probably get the job.

Then the other option is to do a small take home test. Are you ok with that?

The other option is to check references, do credential checks, and simple bullshit screens during interviews, like every other professional interview.

Literally every other profession has this challenge! Programmers are unique only in their seeming inability to realize that they are not unique.

Accountants are not asked to solve double-entry accounting whiteboard problems. Finance professionals are not given take-home banking projects. Only programmers seem to be unable to be reasonable interviewers.

Every accountant attending an interview can produce a certificate from an accredited national group saying that they have taken an in-person, proctored exam on double-entry accounting, which is the 'credentials check' you refer to. There's nothing like it for programmers.

There are plenty of options. Programmers only just have to pick one, or create a new one for all that it matters. For instance, NCEES that runs the Professional Engineering accreditation that every other Engineering discipline utilizes, has a Software Engineering PE. Tomorrow we could grandfather in every Senior Dev with 5+ years experience as a PE and then start requiring it, and encouraging kids to train for it and acquire that accreditation.

One major proctored exam scales a lot better than every company for themselves reinventing an almost proctored exam.

Have you seen the Software PE? In contrast to the other PE exams, it manages to cover everything except actual programming.

https://engineers.texas.gov/downloads/ncees_PESoftware_2013....

That is because it is an actual __Engineering__ exam, not a programming one. Testing for programming skills in an SE exam is like testing a mechanical engineer's CAD skills. I would argue that the majority of "SE" jobs do not actually entail engineering but software development (or construction), which is only a small subset of SE.

The computer engineering exam has tons of the underpinnings covered, with a whole section on relevant math. Where as the software one pretty much only covers process. The one algorithms or discrete math question they have boils down to just 'what is big o complexity'.

I guess the question is what is the necessary "base set" that shows proficiency enough in learning that you could throw someone at a problem and expect them to solve it. Keep in mind that to keep a PE active there are also lifelong learning requirements.

Certainly what I see in the PE today seems like a good starting base. Having some idea of Big O complexity can be a great place to start when working with algorithms, and certainly from my perspective I'd rather have an engineer with a solid idea of Big O complexity trade-off issues than a bunch of rote memorized implementations of classic algorithms.

Also, don't forget that a lot of discrete math and algorithms bubbled up into the Fundamentals of Engineering test that is the prerequisite of the PE test:

https://ncees.org/wp-content/uploads/FE-Ele-CBT-specs.pdf

Which isn't to say that the PE can't be improved for Software Engineering to better meet industry needs, but certainly the current PE still stands as a good starting point that the industry could try to hone to a better tool if they wanted.

And? You think that a certification exam provides more evidence than a university degree? (which, let’s face it, most programmers have).

No, this is a made-up difficulty, caused by the fact that programmers think there’s a silver bullet solution to evaluating competency in an “objective” manner.

You mean a CPA.

My girlfriend has just graduated college with an Accounting degree.

> There's nothing like it for programmers.

What exactly is the difference between a BA (Accounting), and a BSc (Computing) in terms of validation?

Now if you want to talk CPA? Sure, but the equivalent of that would be a Software Engineering (similar to PE/CE/ME).

Apples and oranges.

I don't see how that is actually doable all the time. In my short career I have moved from embedded c / c++ game engine / full stack / DBA / backend. All at different jobs with the previous manager not knowing I actually knew how to do the next job, if they were asked anything other than generic questions I would have sounded like a fraud.

This is not to take away from your point, since I agree with you, but both accountants and finance professionals have stringent licensing requirements which (in theory) could provide a baseline expectation of competence.

That is the reason they get away with not testing them, because they have a universal test they can look to. Software jobs are pretty fluid, so unless we have a massive standard test that cover anything from embedded c to FE with react, it seems that kind of test is out of reach for software people.

Civil Engineering has a huge gamut in specializations and in the last century has seen quite a lot of technological change; programming isn't as special as it thinks it is here. A credential test doesn't have to cover everything, it just has to cover the foundations and get some sense that a person is capable of life-long learning and improvement.

What about jobs like "business analyst"?

Yes. I told the interviewer he could call any of my past managers, knowing they would take the call, and knowing they’d give a glowing report. He laughed!

Well...fuck that guy. There is really no better measure, assuming you have the basic social skills to see through a con job/scam. I mean yah, you could always be tricked into calling someone's buddy or something, but with any depth or tactful questions that is rarely going to fool anyone.

The other, other option is just refuse to do either. I get all my work from friends I've worked with in the last 20 or so years. I mean if a company insists on these hiring practices, and they continue to get bad applications because of it, the probably aren't going to be around for long anyway.

Ok so you are out of the running for this, this is obviously for testing people you have never met before.

Yes. I strongly prefer an offline test, if it realistically wouldn't take more than a few hours.

Yeah we try to have a limit of 4 hours for this kinda thing. We understand that it is the interviewee's free time but when you need to screen 12 people for a job it is not worth a Senior Engineers time for a small startup.

WHAT?!?

Getting the right team is one of, if not THE key to startup success.

Yet you consider it not worth a senior engineer's time to evaluate the candidates for the next hire, and the presumably with whom they will need to work - productively?

Check your org's runway and be sure to have your resume ready when it gets close to the end. I wouldn't expect take-off.

Sorry I assumed that the word 'screen' was universal. This is the first step, it is a waste of a senior engineers time with a candidate that is not even close to ready for the job.

Yes, I suppose 'screen' is a bit ambiguous. Doesn't seem like a pool of a dozen is the first step. I'd want the Sr Devs involved at least out past the final half-dozen if not dozen -- not for 4 hours each, but at least a chat about what they've done, problem-solving approaches, etc.

I wish you luck