"Asking the same questions also makes it easier to compare between candidates and the questions can become optimised over time to ensure they are fair and valid performance predictors."

I've seen this claim getting bandied about a lot lately, but I'm increasingly less convinced. You receive 10 n-dimensional vectors of programming and engineering skill, aka "applicants". You take their dot product with a ten-dimensional vector. Even assuming you can do that correctly and reliably, on what basis are you so sure that the ten-dimensional vector you've chosen has any relationship to what you really need? Repeatably and accurately asking a detailed question about recursion and pointers in a job that rarely calls for it just means you are repeatably and accurately collecting irrelevant information.

I've been chewing on this claim for a few months as people have been making it, and it's not like I'm entirely unsympathetic to it, as hiring does obviously suck. But lately the more I think about it, the less convinced I am this is the path forward. It just seems like a great way to institutionalize bad practices.

I think you need to embrace the diversity of the incoming n-dimensional vectors and pursue a strategy of exploration. Fortunately, they're not all uncorrelated and you do have an idea of what you're looking for. I see interviewing as a process where I'm seeking the answer to the question "What are you capable of?", and I want to seek out answers focused on how that relates to what I'm looking for, and that may mean I thought I was interviewing a dev candidate but I'm actually interviewing someone suitable for QA, or vice versa, or I thought I was getting an academic and I've got an old UNIX sysadmin, and these are just the examples I can quickly give. (What really happens is even more detailed, like, they clearly know their APIs but they're pretty sloppy in their naming, or I started looking at their personal projects for skill and was surprised at the documentation, etc.) It's not an easy approach, but why would we even expect there to be an easy way to analyze applicants?

(Or, to put it in more statistical terms, rather than applying a fixed vector dot product, I approach applicants in terms of a multi-armed bandit, where I'm trying to find their strong points. It helps in this case that the "multi-armed bandit" is actually incentivized to help me out in this task.)

The most important aspect of an interview process is that it is easy to teach. It doesn't matter if you have enough insight if you can't transfer said insight to those who will be on the floor and interview. So no assumptions about common sense of the interviewer is allowed, because common sense isn't common.

So the only thing you really can do is ask objective questions or check technical skills. Thus the choice is not between hard skills or soft skills, but between hard skills and no skills because the soft part deteriorates extremely quickly and becomes essentially useless at scale.

"The most important aspect of an interview process is that it is easy to teach."

That basically begs the question of what I'm saying though; why should we expect that there is an "easy to teach" process that "works", for some useful definition of "works"?

Our desire for something to exist does not mean it does exist. I think a lot of the rhetoric around "fixing" interviews is making this fundamental mistake, where what is desirable is laid out, and then some process that will putatively lead to this is hypothesized, but there isn't a check done at this phase to be sure the desired characteristics can even exist.

It may be that evaluating human beings and whether they can fit into technical roles is just fundamentally hard. (I mean, when I put it that way, doesn't it sound almost impossible that it's untrue?) While acknowledging fundamental difficulty is not itself a solution, it does tend to lead one to different solutions than when the assumption is made that there must be some easy, repeatable solution.

I can sit here and wish there was some easy way to communicate how to be a full-stack engineer in 2019, but that doesn't mean that such a thing exists, and anyone trying to operate on the presumption there is is headed for the shoals at full speed. People who know such easy things don't exist may still wreck themselves but at least they aren't guaranteed headed towards the shoals.

(Or, to put in another really technical way, the machine-learning-type bias that somewhere in the representable set of interview processes there must in fact be a good solution is not necessarily true, and think there's good reason to believe it's false.)

"Also, it's never obvious how much someone contributed to their portfolio."

You guys are absolutely correct about the reference questions. I can find no fault with that.

However, there's absolutely no question that I'm the sole contributor to all my code. Most of my repos have only one contributor (Yours Troolie). A couple have been forked off.

I do link to a number that have been turned into larger projects; with multiple contributors, but there's no question that's happened.

Also, those are my old PHP repos; not the ObjC or Swift ones.

There's ten years of commit history in the repos. You could run crunchers on them to do things like develop velocity and productivity metrics.

I'm also quite aware that my case is unusual. Most folks don't have such extensive portfolios, and are much more of a "black box."

> However, there's absolutely no question that I'm the sole contributor to all my code. Most of my repos have only one contributor (Yours Troolie). A couple have been forked off.

A bad actor could easily just have someone else write the code or copy it though.

I think that realistically, the only person who could fabricate a repository commit structure in a way that wouldn't be clear that is what they had done, would be someone who can program. If I am taking a working program, and slowly breaking it up into pieces starting with tests, and simpler components, fabricating the addition of new components each commit. That is work in my opinion, only an actual programmer could falsify, and if they are an actual programmer they would have actual repositories.

Yep, exactly.