I recently was flown to Austin for an interview. The entire reason I was brought down was because of Python ML projects on Github. The technical interview consisted of nothing, and I mean NOTHING but super advanced SQL questions. (over the phone, I had specifically stated I hadn't used SQL in years) My expectation was that I'd be interviewed using Python and asked to perform ML related tasks.

When I mentioned that I thought that the purpose of me being brought there was to add an ML dimension to the team, which was far too data analysis (using SQL and some R)focused and with no Python expertise, I was given a blank stare.

Then it hit me: the guys interviewing me didn't know how to do any of the stuff I had been brought down (by higher ups who weren't in the room) to do, so they weren't evaluating me on it. They evaluated me on what THEY knew. It's the equivalent of Peyton Manning being asked to evaluate a linebacker, and demanding that the linebacker throw passes downfield.

The highlight was when one of the guys (typical pony-tail neck-beard type) pointed out that an alluvial flow diagram in my portfolio of data visualization projects wasn't "Tufte-esque". (It was a gross misinterpretation of Edward Tufte's commandments on his part, but am I really going to get in an argument with the guy interviewing me?)

It was clear that the guys in the room wanted someone who knew what they did and thought like they did. A brilliant recipe for getting a homogenous team with no diversity in skills.

What an epic fucking waste of time. The best part? The company only has 150 employees, and a recruiter just contacted me for a position on another team that uses Python. She was unaware of the fact that I was there a month ago. I told her that they should have thought about that when they flew me down the first fucking time.

> I told her that they should have thought about that when they flew me down the first fucking time.

Thanks for that!

On the other hand, most likely the other Python team would grind you on Python and again nothing on ML, and of course your Python ML projects suck because "blah blah blah"

Maybe if I am in a similar position, I should demand compensation for my time. That way I'll at least get to visit friends and see scenery around Austin. I wonder if a demand for compensation (besides airfare and stuff) would fly...

It would not. The only thing I could imagine countering is: Fly me to Austin for two weeks, put me up in a hotel, and hire me to do a project at $x/hr. If we both like what we see, let's talk about a full-time position.

Frankly, I think this is anti-candidate behavior if the employer suggests it, but if you actually want it, they should be willing to do it if they can vet you otherwise (github, etc).

I usually just stay longer on my own dime. The company doesn't care whether they pay for flying you back immediately or two weeks later.

Ugh, what a wasteful mess. The worst part could be my lack of surprise in reading your anecdote.

Better analogy is that classical musician is given sheet music to pop song and is told to play it on his favorite instrument. If he is good musician, he should be able to play it reasonably well even without practicing. There are musician who play extremely well but only after practicing one song for a long time - this type of interview sucks for them but they are minority. Most musicians suck with or without practicing so this type of interview is a good filter. Not ideal but no one came up with better one that scales up to thousands of applicants.

In an audition, musicians are usually asked to play a mix of prepared and sight-read pieces (though sometimes the sight reading is from standard repertoire, so they may have encountered before). Both are helpful in discerning how qualified the musician is. And sight-reading is incredibly important for most jobs musicians get hired for - in most freelance situations, they'd get a 3 hour rehearsal, or essentially a guided run through of the performance, then they play the gig. So that may not be an accurate analogy, given that writing on the fly isn't part of the programming repertoire - most programmers are allowed to think before they write anything down. Also musician's don't write - performing is very different than writing- so perhaps a composer would be a better analogy.

Maybe it would be a more accurate measure of ability to ask software engineering applicants to read code and explain what it does, or to judge them as you'd judge a composer, by their portfolio.

That would be an interesting way to apply something like re-captcha: ask them to fix two bugs in open source applications. One bug that you've actually already fixed, so you know how hard it should be and can calibrate; the second bug is open.

How about just asking them to code something - anything - in say, a couple of hours?

To make this useful for an interview, you have to set it up so that people can't submit things they've coded up before in a much larger timeslot.

Not all codes are created equal.

Agreed; the idea of hiring a musician without hearing them play would be madness. Not handing me a recording, not discussing intricacies of music theory, but actually playing live. And yet, a talented studio musician may suffer from performance anxiety in this situation. What to do?

I think the best thing is to ask them to warm up with something basic like a scale, and let them work their way up as their comfort level increases. I knew some people who were particularly pleasant and easy going, so they were able to relax people suffering from performance anxiety.

There are interviewers who treat coding interviews as a high pressure, adversarial exercise, though. On the other hand, those sorts of people are rarely a joy to work with.

Performing a piece of music in a high-pressure situation is exactly a professional musician's job, so an audition is quite appropriate. Another analogy could be to ask a professional composer to play one of his pieces on piano rather than listening to a competent orchestra perform it--you're using one thing as a proxy for another. Many composers would ace it--they're great players. Some aren't, though, so you'll hire only composers who happen to be great performers. It's a dumb analogy in another way, though: you only need to know whether you can work with a composer, their work should stand on its own. I suppose if we engineering types all wrote code that went into public repositories the same might be true of us, and we wouldn't be asked to perform like trained monkeys.

In a lot of cases, it's like a classical musician being asked to make a violin.