This point cannot be underemphasized. "Thinking aloud", not just in front of another person but under a very stressful situation (for most people) that also basically almost never occurs otherwise in daily life -- is a specific metacognitive skill that for many people can only be (even adequately) learned by either (1) careful training or (2) surviving many repeated failures, and finally getting at least partially acclimated to these kinds of confrontations.
Yeah sure, perhaps some of the people who "bombed" that calendar test really were the walking fraudsters the interviewer (who knows they'll get to stay in their well-paying job regardless of the outcome of said interview) makes them out to be. But most are probably simply nervous, and due to a variety of psychological factors (imposter syndrome, among others) are simply momentarily blanking out, and having suffering from a very common form of mild anxiety attack which makes the problem seem much more complex to them (at that moment) than it otherwise would, under more natural circumstances.
I've literally started nervous people with "naively print the string 'hello'"... After warming up, the best one eventually completed my max-difficulty questions.
I would add that it takes significant experience and preparation to interview people effectively. This is a difficult-to-learn soft-skill that not everyone has.
Too often folks just get dragged into an interviewer situations with no planning or coordination.
Is there evidence for this claim? I'm inclined to believe it but that's experiential. It's also something I've never had to work to acquire, but that could be a cultural thing (my family and friends talk a lot about thinking) or a personal history thing (I have a minor learning disability, and it has forced me to develop strong, conscious meta-cognition).
It's not a given, tho, that being able to talk about thinking is distinct from being able to think and being able to talk.
But talking through a problem forces you to use verbal reasoning. A lot of programming can be done well with non-verbal reasoning skills.
Personally, quickly sketching some timelines on a piece of paper would be the fastest way to get the correct checks. Talking it through would force me to convert my mental visualizations into words, and that'd make me stumble.
people who "talk to themselves" are labelled idiots. mental dialogue is supposed to be mental.
call it social conditioning if you will. spelling out your thoughts for someone else to hear them just so they can take note of them for evaluation is the polar opposite of normal human interaction.
The problem with the modern interview process is that not only elevates the later approach way out of proportion to its actual significance -- but basically makes in the centerpiece of your interactions with the company and potential team members, on that fateful day.
Interviews aren't ordinary job situations: they are contrived situations where people are making life-changing decisions based on a very short interaction.
Agree mostly -- just that I guess I've fallen into more abusive environments than I should have allowed myself to over the years, where the stress levels have hit will into the 7-10 range.
But in healthy environments -- even when I really did screw something up (sometimes quite significantly) -- it's never above the 4-5 range.
Interviews aren't ordinary job situations: they are contrived situations where people are making life-changing decisions based on a very short interaction.
And pretending that they can read the tea leaves, and make that kind of assessment as to "whether this guy can handle a medium amount of pressure", to boot.
you don't know the circumstances that 30 year old programmer with a 2 year old child at home is facing. You could also be like certain firms and immediately nix him on the "merits" of ageism.
A female programmer might have travelled 2000 miles to just receive an abstraction question out of left-field over an irrelevant concept to the job. She flew out on this uncertainty, and she's in a rather discriminatory field.
26 year old dev has probably built CRUD apps all his short career. and, you're asking him to implement (isBinarySearchTree Boolean) on a whiteboard on the spot.
A choice, conscious or not, to dedicate time to raising a child at the opportunity cost of myriad other ways to spend time is not related to ageism at all.
Hear! Hear!
One of the things that bothers me a lot about HN is the near assumption that if you're a software engineer and not in SV (or have never been in an SE job in SV) - you don't count as much.
I'm biased, I will admit: I've never held a software engineering job in the Valley. That opportunity has never occurred, nor have I tried to pursue it. I live and work in Phoenix, Arizona; at my age and position in life, even if the opportunity did present itself, I'd have to think long and hard about it. It would likely be a situation of me leaving my family to go work there, as relocation would likely not be a realistic option unless the pay was over the top (I am not going to move back to an apartment; my next house better be at least as large as my current one, and include a larger yard and/or shop space before anything else).
Here, in my current position, I am not only the oldest developer on the entire team (and I am only a few years younger than the owner of the company), but I also am one of the few who doesn't have any children. My wife and I made the conscious choice not to have kids a long time ago, and we are constantly glad for it. In fact, she thanks me on a monthly basis for not "knocking her up".
Honestly, we'd probably make terrible parents, and we're pretty selfish as a couple. We know this, so why bring children into the mix, right?
That said - here in Phoenix (where it's actually difficult to find competent software engineers for any position), I've never seen or experienced there being an issue if you have kids; every employer I've worked for has been extremely flexible.
In fact, both at my current employer, and my previous one, they encouraged you to leave when your day was up. If the end of your day was at 5pm, they didn't want you stay a minute more if there wasn't a really good reason for it. Like things would have to be well and good on fire for that to happen; anything less was "go home, get some rest - it'll be here to fix tomorrow".
The attitude out west is "You want to earn enough to raise a family? You should have thought about that before you had kids".
It also seems to me that the people who have the most trouble with work/life balance in development jobs, at least out here, is that they are too afraid of saying "no".
absolutely true (at least in my case). I'd rather do that than end up on the other end, though. Without stronger protections for employees, I'll hold out for a stronger financial position on my own before I start negotiating hard about "work/life balance".
When I've been through such phases, once I realized that I could set reasonable expectations, it was never actually a big deal. Eventually I even started taking my vacation days, nervous that my job wouldn't be waiting for me when I got back, but that was really all in my head.
I've been in many stressful situations at work - none of them involved finding algorithmic solutions.
It's the idea that this kind of pressure can in any meaningful way be simulated (and the candidate's response adequately gauged) using the shticks and routines that people try to do in the current standard interview process that people take issue with.
Not quite all about algorithms, but that was part of it :-)
Also found that the probability of someone finding the porn in the demo by randomly pressing remote buttons pretty much approaches 1 as number of button presses approaches infinity...
Not once in my career had I had to fix a problem in an hour, let alone 10 minutes.
I have had senior people get mad at me when I couldn't give an answer during a meeting - that's the closest to an interview type scenario where you need the answer now. But even then, I did not stress that I'd lose my job, which is similar to the stress the candidate is facing.
Yeah, but if so it's usually of the form of "oh shit, forgot to chmod this little bash wrapper".
To the extent that it's of the form of "Solve this cute little dynamic programming problem I heard about from some company where everyone is like, gosh, oh so smart! In like, the next 5 minutes, k?" -- well, basically never.
It's like interviewing a doctor who will be someone's primary care physician. Give him/her a set of symptoms, and fail him if he doesn't diagnose it inside of 10 minutes. And then justify it with the expectations in surgery or ER.
I'm sure good at fighting database fires though.
This probably is at the heart of the answer to the question "how could this programmer with such a long resume have botched this interview so badly?" It would be insane to believe they've never added value to any organization and have just been coasting along all this time. Maybe your interview sucks and isn't good at finding value.
I know you did not mean it this way, but in some ways that's even worse. If that type of thing were routine enough to merit asking every interviewee that question, I would seriously reconsider working there.
Armies train people by having them run obstacle courses with guns being fired all around them. Do a "realistic" interview, then, to really test how someone does "under pressure". Have them sitting alone in the room, and a screaming angry manager runs in yelling obscenities at them about how the production system is down, it's costing the company millions of dollars, and if they want to have a job they'd better get it fixed ASAP!
And have that same manager stand over their shoulder the whole time, yelling yet more obscenities. If the candidate doesn't get it fixed in, say, five minutes, then obviously they lack even the most basic qualifications to work as a programmer and you can reject them. Heck, save money and don't even do it on-site -- get their phone number and do it 3AM on a weekend, just like a real on-call situation!
Or... maybe the whole "under pressure" thing is just an excuse people use to cover for the fact that they like making people squirm, enjoy the feeling of power they have from knowing someone else's future is in their hands and that person knows it, and want to savor by making the trained monkey stand up and dance on command... or else.
Unfortunately, people who like the "under pressure" idea of interviewing seldom realize that sooner or later they're going to be the monkey and someone else will be the organ grinder.
Sure -- it's just the tenor (and for some people, sheer intensity) of the psychological stress experienced during interviews is, for many people, basically orthogonal to the stress of real-life work situations -- or even genuinely dangerous (even physically threatening) situations in life, otherwise.
With the latter, it's like "Oh shit - the client's gonna get real pissed if I don't figure something out real quick".† Or even: "That guy looks like he's about to lunge at me - I better think of something!" For which your brain and your glands have benefitted from millions of years of evolution in support of mechanisms for pumping just the right kind and amount of juice into your system to "figure something out".
But with interviews it's more like: "This person's evaluating me, using some hidden criteria. Given that the problem is slightly tricky, I literally don't know if they're expecting to power on at all costs, or, secretly, that I simply admit that I don't know where to start just yet. Not only that, something tells me he's not stating the problem quite correctly -- they do that, no all the time, but way too often in these kinds of interviews. And on top of that he's just being plain overbearing... and now he's starting to fiddle with his phone. On top of starting 15 minutes late. Like he never really wanted to talk to me in the first place. If I ask too many questions -- or even just one instance of the wrong kind of question -- I'm screwed, and gonna have to hit my inbox for leads again. Oh great, now he's starting to offer 'hints', completely derailing my flow of thought and whatever self confidence I thought I had. Fuck it -- I just should bail and go work on my personal projects. No one will be paying me $150k but at least I'll be learning something."
† Where "the client is gonna get real pissed" is roughly equivalent to "if I can't get these rocks to flake in just the right way, we aren't gonna have any more arrowheads and were' all gonna starve!", to a first order approximation.
understatement of the year.
I am not saying it is right, I am just saying at some shitty places it is.
That's a high-pressure situation, so if you're hiring for that, it might make sense to interview under pressure.
I have worked on systems that could cause operator death or disfigurement or damage... they are tested quite thoroughly in general and have reasonable failsafes and redundancy. It's not like they are coded on a whiteboard.
being on site at a customer facility while there is a ton of noise does stress you out a bit but most of the coding and testing is done in the office.
> That's a high-pressure situation, so if you're hiring for that, it might make sense to interview under pressure.
I think the jobs that have that kind of pressure are rare and interviewing like that should be the exception not the norm.
The problem with giving technical interviews is you are testing for someone who is an extrovert that can bullshit under pressure. That's not what you want. You want someone who is smart and can solve problems with a compiler.
I have not interviewed many people yet, but already during one lunch (!) interview (where I don't ask any questions), a guy was so proudly talking about his achievements at his previous job in quite generic terms, that I've got an impression he was teached the whole story, rather than actually lived it.
Later he was not hired. According to (trusted) others in the interview loop he did not know how to do any basic shit.
Having said that, let me be clear. This isn't a good tool if all you have to do interviews is a manager. They probably won't be able to tell the difference, especially if they are just generic managers with little to no technical experience. They aren't the type you want giving technical interviews.
Asking about favorite projects is easier for extroverts that can bullshit under pressure than technical interviews.
Do you think introverts don't like to talk and to brag?
Ask an extrovert bullshitter who couldn't care less about Star Wars about Star Wars and you can easily tell the difference. (as long as you yourself aren't an extrovert bullshitter who couldn't care less about Star Wars)
Edit: that should be 'overemphasized', of course.
I don't think they are fraudsters, they are simply not too good programmers who can't solve simple problems by themselves. Many dudes believe themselves to have technical talent just for memorising done basis. That does bit imply problem solving ability.
Momentary blackouts happen - rarely.
I am about to start work as a senior engineer for Apple with 4 1/2 years of experience, after passing two back-to-back onsite interviews with them, and I hardly have studied specifically for technical interviews. I possess an MS in mathematics.
Don't presume anything about an individual just because the person happened to not answer a particular question - the person could be actually much smarter than you. It could have just been a bad day, or a number of factors.
But then there are people for who practicaly any question beyond "lets talk in general about what you like and how passionate for technology you are" is unfair and causes black out entry single time. Thay is entirely different.
It is super important to try to ease a candidate into being comfortable to get the important information you need as an interviewer, as interviews are inherently stressful for candidates, and you almost never want to be testing a candidate's stress coping abilities.
"I dunno - they put me through this stuff, or some variant, on my way in. So now I guess it's my turn to dish it out on the people trying to climb in after me" seems to be about the order of reasoning in place behind these "methods".
† As in, was an open problem in the literature and/or folklore for years before some (now) highly respected came up with the first reasonably adequate (but like as not, overcomplicated and/out outright flawed, at the outset) solution. We know the "usual suspects" that get asked -- I don't need to list them here. But ff you genuinely think that any candidate you can (reasonably) expect to find in response to your muddled job postings will genuinely be able to solve these problems for you at the whiteboard for you -- from the void, as it were -- as opposed to regurgitate carefully practiced solutions they found from cobbling together lists of "problems Google and FB like to ask" -- you're seriously kidding yourself.
There is nothing to suggest that parent poster has bad social skills or that his job posting was muddled.
Do you know "majority of people" or have interviewed them?
Have you yourself faced an interview question at an abstraction level far below your area in this field?
Yes, I have faced questions I could not answer. I did my best and that was it. That is however not blackout - that is me not knowing the answer. Blackout is when you temporary can't recall.
But that's the thing -- it literally takes just one person out of 5 or 6 to say "I don't know about this guy" to get the rest of the team to pass. No matter if he didn't seem all that well prepared or interested in you, you guys just didn't click, or you were exhausted because no one thought to let you get coffee or use the restroom.
This never happens in normal work, even when working under pressure with a deadline that has to be met and not enough time to make it.
It's just a completely different setting and purpose. In one case I'm solving a problem for the sake of being judged and that judgement may impact my whole life for the coming years. In the other case I'm solving a problem because there is a practical need for it and I'm just doing my job.
It has nothing to do with being a good programmer or not, it has nothing to do with my actual ability to solve a problem either.
Funny thing -- these folks have, y'know, actual data on this stuff:
http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w...
Choice quote:
As you can see, roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.
(assuming the list happens on the same day)?
What I like about pklausler's example is that it allows for people to find edge cases without too much technical knowledge. If I have a candidate who doesn't ask about the inputs (e.g. "are the appointment sorted?"), I'd be wary. Engineers are often given vague specifications they need to clarify or account for.
Unfortunately, this can also occur because the interviewer isn't clear on their own goals in the first place. They asked the question before really knowing what data points they want to come away with.
One could even argue that relying on the order of the items in this case is going to result in a worse design overall (it's one additional thing the caller can get wrong).