Interviewing in a sense is kinda like dating, one way to really screw things up is if you're both looking for different things. If you're both trying to get into a fling, then yeah I guess lie your pants off and have unreasonable expectations of each other. If you're looking for long-term match, just lay your cards on the table and if you match, you match.
Here I thought it was their high false negative rate. I should have known it wasn't the google process.
This is an excellent point. If someone does this I view it as a signal that they are intelligent, rational, and honest.
"Tell me about a time in which you..." works very well. If they resist answering the made mistakes questions we prompted them to understand that an honest answer is important. If they could not do that then they were not considered.
These were not developer roles but most were very technical / analytical roles for an organization that paid well and candidates knew up front they would be interviewed this way. In addition some jobs were "cased" where you had to solve a problem and talk about how you solved it to the interviewer. Great fun when I went through the two days of interviews. I ended up doing maybe 200+ behavioral interviews over 6+ years for the company. I saw this approach work time and time again.
Some people could not answer for themselves: "the team did..." "They did...". When pressed we usually discovered that the candidate did not have the ability to get the job done themselves.
These types of questions probably cause me more stress than any other. Being asked to think of a situation where you felt a certain emotion or had a certain mental experience is not actually that easy for everyone. My memory is terrible at that kind of thing.
It's like my memory lacks a necessary index for what you are asking. I can start going through every major work experience I've had in sequence and filter out the ones that fit, but we might be sitting there in silence for 2-3 minutes which gets awkward enough in itself.
And it's not an inability to remember mistakes specifically, I mean the old "tell me about a time where you solved a problem with no prior knowledge of the domain..." gets me. I solve problems like this ALL THE TIME. That's a skill I have. Fetching those times from memory is not a skill I have.
Luckily I've had these questions in enough interviews that I generally try to have some examples in mind.
How will I know if this person is good at, say, sales or biz dev in a vertical with long lead times? I can't vet that skill set directly; I can only ask the candidate about it. Behavioral questions give me something to cross-reference in the absence of on-the-spot performance data. So do case questions, which (at least ostensibly) tell me how a candidate thinks, and how she organizes her thoughts. (And of course, domain-knowledge questions help me assess the depth and quality of a candidate's experience in the field.)
Behaviorals aren't perfect, though. They can be gamed. It's not too much of a stretch for a candidate to memorize a half-dozen anecdotes to trot out in response to any question that begins with "Tell me about a time when..." (Nevertheless, you'd be surprised by how few people do that sort of prep work.) Canned responses are especially common with candidates straight out of school or grad school, where they are often trained in how to answer behavioral and case questions.
From my experiences with employers in many parts of Europe, they value partial contributions to teams more than complete individual achievements. Talking to much in "I did..." sentences instead of "We did... (...and I helped with...)" can come across as self-centered, which often is considered a negative trait here.
Personally I'm pretty tolerant of a lot of BS, depending on the person. Sometimes it's just a cognitive quirk that someone uses BS to search around their brain see if they can trigger some association that lets them remember, or discover that they really don't know. Saying "I don't know" immediately is often the wrong thing to say (especially in interview settings) when the real answer is "I don't remember, let me rediscover / look that up", but not many interviewers let you look things up or will mark you down if you say you'd need a reference for X though you studied X some years ago in college like everyone else and might work it out (after wading through a bunch of BS and false-starts in general) if you had unlimited time and no pressure...
One is, people want to demonstrate what they know about the topic, even if they can't answer the specific question. If you ask me to explain elliptic curve cryptography, I can tell you that it's useful because it uses smaller keys then prime based RSA crypto, and that's something I'd want to mention to indicate that, despite the fact that I'd need to look at a book or writeup to actually implement elliptic curve cryptography and I'm not sure I can verbally explain the math behind it, I at least know something about it.
The second is, at the limits of my domain knowledge, I may not know what I know and don't know. Limits are fuzzy, and I may feel like I have a pretty good understanding of a linux file system from a user perspective (permissions, inodes, etc) but I couldn't implement one off the top of my head and I'm probably missing some important components. As an interviewee, I may have incorrect or incomplete knowledge, but not know that. That doesn't mean I'm bullshitting intentionally, I'm just trying my best to answer the question I think you are asking.
Also I have to think that other developer cultures may have different values around these ideas then my own. Maybe the cultural expectation is to show confidence in my ability; I know that confident individuals are more likely to get hired or even apply for jobs, and that often job requirements are a screen for confidence.
Sure, I'd love if I knew every gap in my knowledge and every mistaken understanding I have, but I don't know how realistic it is to suggest a deliberate deception on the edge of a person's knowledge.
And that's setting aside whether the interview process is a good predictor of workplace behavior... I think we all know people who are excellent devs that can't write on a white board for shit, or who sit down for a good think for an hour and then come up with a perfect solution but aren't going to be able to write and simultaneously talk you through a solution to a 15 minute data algorithm question.
That said, I think you balance assumptions vs knowledge in order to be effective. Like you assume that it's in the man pages, and the man pages describe the code that is being used on your system, but it's entirely possible that your version of vim or whatever is different then the one described in the system wide man page.
A developer who can't accept a challenge that stretches his/her boundaries a little bit is probably not a developer you want.
But, once you do that setup, everyone is eventually going to say that they don't know what they don't know. It's exceptionally rare for someone to be able to delve into the implementation details of a couple of different programming languages (of our choosing), cryptography, distributed systems, relational modelling, CS theory, and a variety of algorithms and data structures.
So "I don't know" ceases to be much of a signal, since everyone says it a few times.
In fact we prefer to see the opposite, where people are willing to tell us the bits of info that they do know. Even if you don't know how two phase commit is implemented, knowing what problem it's trying to solve will allow you to research it when the need arises.
Because an interview situation is so unlike a normal workplace environment, I don't know any reliable way to translate their willingness (or otherwise) to say "I don't know" in the interview, to on the job behaviours. Either you'll coach everyone into it, or you'll be letting the interview pressure (potentiality) distort their behaviour so much that it's too hard to derive a signal from.
You're welcome to guess or describe how you'd think about something, but you should be willing to clearly state when you're really out of your depth. It's important to note, though, that this isn't done adversarially; it's usually just a natural conversation that goes down a particular rabbit hole.
I talked about it openly but he kept pushing. In the end my feedback was that my "time I've made an error" was not very impressive.
Like... wtf? I find these questions very weird. I never know what they're expecting as an answer.
What experiences triggered that learning process, or did you just take orders from someone else with a more problem-solving oriented worldview.
The same applies to technical work. We're not perfect robots, we are given the freedom to figure things out, so the goal is a refinement of the "program" in our minds. The more effective/sophisticated that program is, the more rational we are.
In other words, it's possible to be smart but not act smart, and it's possible to act smart while not being exceptionally intelligent.
I'm basically looking for self-awareness of one's own cognitive limits and evidence of a meta-process that seems likely to result in better decision making than whatever the raw intellect would offer.
Another thing is that my current environment IS very CTO pushy taking order and doing his view. Why stating this counts against me? It's not like everyone can take leadership in the tech stack / implementation.
I am a 100% sure that I would get into his (big) company and just follow orders as well. Of course having my implementation and my personal touch every now and then but still I wouldn't get to make architectural decisions or choose the tech stack.
Unless you're working in a 100% microservice architecture with hundreds of small people team deploying separated stuff I don't see how that can happen.
I would never hire someone who couldn't honestly speak to making mistakes and being wrong. We're human, and your ability to be self critical and learn from your mistakes is one of the single most important aspects in continued professional (and personal!) growth.
It should never be difficult to speak to a time you've screwed up. No one rational thinks we go through life with a perfect record, and no one has done it. Be honest.
In other words, is the candidate someone who has a rational meta-process going on about his/her decisions and is able to refine his/her approach based on learnings. I'm ideally looking for small things that show clear evidence of that kind of thought process.
As for body language, I agree, and I think my choice of the term "poker face" was a bad way of describing a certain kind of evasiveness which can indicate bad motives or bad people skills, either of which can make the person a bad hire.
As a practical matter, how do you detect this kind of "evasiveness", and differentiate it from (say) plain old introversion or nervousness?
Suppose you're having a conversation with someone and in 15 minutes one of you chosen at random is going to have to trust the other with his/her life. What kinds of signals indicate that the person is trustworthy, understands the potential future, and is accurately representing him/herself?
There are some people where I never feel the kind of connection I'd want to feel in that scenario, and others where I feel it fairly early on in the interaction. Just a kind of straightforwardness, not extraversion or social comfort.
That is certainly a risk with any notion of "culture fit", but any time someone who cares about culture is interviewing someone that fit is being evaluated to some extent.
I think this is a super important characteristic for a great employee to have, but I don't know how reasonable it is to expect to see much of it in an interview. In an interview setting, the candidate is under a lot of pressure to come across as "perfect" in the sense that they want to appear to meet all of the expectations the interviewer has for them.
Asking about mistakes they have made, or putting them in a position to make a mistake during the interview itself can just be really stressful. Candidates that think you want them to make no mistakes will feel they are doing poorly. Candidates that know you want them to be comfortable being wrong—whether or not they actually are—can easily game this.
> - poker face -- does this person seem to be playing it close to the vest in the interview or otherwise have very low levels of openness? This is a big warning sign and an instant do not hire.
One of the things I've heard from HR/recruiting folks is that it's not a good idea to use body language as a signal. There's a lot of cultural and personality variation around it. You can end up with an unfairly biased opinion simply because they come from a culture that is not very demonstrative, or they are introverted or otherwise have personality traits that lead them to not display a lot of emotion or energy.
I agree with all of the characteristics you list here. It's a great snapshot of what I like in a good coworker. But, man, accurately detecting them in an interview seems super hard to me. But maybe I just don't have much confidence in my own interviewing skills.