That's what's so mind-blowing to me about our terrible process today. It aggressively tries to find and dispatch unqualified candidates, so much so that it forcefully repels talented people. And it fails miserably at screening out people who can't code! It is the worst case scenario of processes.

It's also really important to remember that the problem is People Who Can't Code, not "People Not Smart Enough To Code". Time and again I've "worked" with people who could very effectively critique my own code, who could participate in design meetings down to the bits-and-bytes layer, and, having taken responsibility for actually implementing some portion of those designs, could not commit usable code to save their life. I don't know what the deal with these people is, but they are out there: plenty smart, totally fluent, and completely ineffective.

I was talking to Erin this weekend about the phenomenon and realized that these people not only exist, but are actually favored by the way the market works. A smart person can last many months in a dev role before anyone realizes they're ineffective. Then, it takes many, many more months to manage them out of the team. By the time the story plays itself out, the bad team member has more than a year on the job, and when they parlay that resume line item to another prominent job elsewhere, their previous teammates aren't outraged but instead relieved. They end up with gold-plated resumes and tons of credibility.

1) Young, fresh graduates who still remember well the CS material. Or those who have a lot of free time and can study for these interviews (for example look at this Quora link http://www.quora.com/How-much-time-did-you-spend-preparing-f... the first reply mentions spending 28.5 hours per week for 5 weeks, which is 142.5 hours total) From a senior software engineers perspective that's a ridiculous commitment for something where an arrogant asshole interviewer can shutdown a whole onsite interview process because they're having a bad day or have some favorite question they think is the One True Question for determining if someone is a software engineer.

2) The software industry tends to overvalue youth. It has a very high salary at start, but also a very quick salary cap. And most companies have no technical ladder, and even those that do it's a joke compared to the manager ladder. All of this is not surprising. Since the industry is so heavily biased towards hiring and rewarding youth, older experienced engineers are not valued.

3) Senior software engineers have heavy pressure to leave the software field. I'm feeling this now. I'm a senior at my current place and my salary is basically capped. There would be little to no benefit going to another company. And even if I wanted to, I have a family now and don't have the free time to study for software interviews. My options are basically go into management, consulting, or start a business. I have no desire to start a business or go into consulting because I simply don't like doing those business related things. I want to be an engineer not a business person. But if I just stay an engineer, I will probably get laid off someday due to ageism. And if I'm an old engineer who is laid off, job mobility is basically gone due to ageism and the CS heavy interviews that I don't have the time or motivation to study for. So the only safe path is management. A lot of senior engineers face this decision. Which again reduces the number of mentors and experienced engineers to teach the new younger engineers.

4) And doesn't anyone notice just how wrong whole concept of studying for an interview is? There's a whole book publishing industry dedicated to technical interviews such as Cracking the Coding Interview. The technical interview has become like the SAT. Many could just study to pass it, but be horrible engineers.

http://www.conweets.com/tqbf/gayle/

(I'm having a pretty bad day, which I hope doesn't show through too much in that summary).

In that context, "give take-home tests" and "ask better questions" sound like the exact opposite of good answers.

I think this is a problem that's unique to very large corporations. Not only do thousands of people interview there, meaning questions and answers are likely to be traded verbatim online, but there's also actually an incentive to cheat your way in: you can hide out for 1-2 years and acquire a horde of cash and resume fodder.

Google have actually made this problem worse for themselves, too. By overvaluing their interview process, they've made cheating even more lucrative. Cheat your way into even an offer there and you have something valuable to put on your resume!

I suspect you haven't seen a lot of cheaters on take-home tests before because at a company like Matasano, there's little incentive to cheat your way in. You certainly can't hide out in a corner doing no work there.

Google is no more a proof of the value of algorithm tests than Github proves the value of manager-free organizations.

I was a bit horrified at the acquisition coaching comment. I did not realize that happened but of course it makes sense. Can't manage an acquihire if your devs can't pass the algorithm tests at the big acquirers. Shame.

It is actually more profitable for me if I say that these algorithm tests are BS and completely study-able, and even the worst engineer can pass them. That easily translates into "here! buy my book and even YOU can get a job at Google!"

It's actually a much harder sell when, really, it's more like studying helps someone perform better, but only to an extent.

Oh, and the acquirers encourage studying and prep, just as most of the top tech companies do for their normal dev (and non-dev) candidates.

To be clear: my experience is that the algorithm interviews, when done properly, work reasonably well to identify intelligence/problem solving and coding skills. Those are more important at some places than others. They can work well for certain places -- but not at all. I do not think that all companies should use this process. It's not right for all places, but it's reasonably good for some.

As for Google being proof of the value of algorithm test: I think you're pulling that from a twitter conversation very much out of context.

Context: Some people were arguing that the algorithm interviews offer zero value - that you'd be at least as well off, if not better off, hiring at random.

If they truly believe that (and they appeared to be sticking to this argument), then it's absolutely relevant that not just Google, but basically all the top tech companies, use this hiring process. Could a randomly selected group of engineers build the technology at companies like Google, Facebook, Amazon, etc? I suppose that's what they must believe, and I find that pretty absurd.

If they had argued that these interviews sort of work, but aren't nearly as good as some other interview processes, then you're right that the company about Google and other companies wouldn't be relevant. And I also wouldn't have brought it up.

Given that there is no correlation between job performance and interview score, are you saying this based on your gut instinct, or do you have objective data somewhere?

My best estimation is that the Google interview process is just a glorified fizzbuzz presentation, and that the hiring decisions are more or less made by arbitrary factors, including the nebulous "culture fit".

While it may be amusing to work through the details of doing a merge sort on a linked list in 30 minutes, it's certainly not germaine to the quotidian experience of an SWE at Google.

That's not a correct assumption. I suspect that you are referring to the (infamous) Google studies on this that were alleged to show this. The articles about this got many details wrong, and the study was fundamentally flawed. Remember: they only studied the people who did very well. That's like finding no link between exercise and health when you only study people who run regular marathons.

Merge sort on a linked list is a bad interview question, so I agree with you there. Good questions are problem that actually make you design a new algorithm.

I disagree with your assertion that lambasting Google's hiring practices is an easy sell in the market. The last time I tried I was dismissed out of hand by a talent consultant.

That... wasn't my assertion at all.

This has been going on for ages. A lot of coding bootcamps and similar things are just interview prep programs. A lot, I mean a lot, of people I have worked with can nail interviews but can't do real work for shit.

Unfortunately to land a job, you have to go through this process. Developing interview skills is parallel to being able to do actual work.

How? If the person is not delivering stories it's pretty obvious. I've seen a few people get the boot after three weeks.

Team accomplishments can be embellished, stronger team members can cover for the weaknesses of weaker team members, but admitting that a decision you made was incorrect gets punished. It's obviously better to hire someone competent in the first place, but the next best alternative, for the manager, is to live with their bad hiring decision.

As engineering managers, we were expected to "manage out" about 10% of the lowest performing staff on a yearly basis. The way that worked was basically that the 10% under-performers were to be put on strict personal improvement plans, and were under no circumstances given raises at all, and it should be made clear that no raises would ever come their way unless they improved.

That was it. Sounded harsh if you were used to 10% a year raises, as many of the good performers got. Not so harsh for those who were used to live in fear of getting "exposed" and fired.

And as a manager, if I were to designate a lot of my team as under-performing come review time, it'd reflect badly on me, so managers also had an added reason to push for good reviews for their reports: You'd end up getting paid more by getting good reviews yourself if your reports were helping create a good impression through rankings that you as a manager could trivially fudge (we did "360 degree reviews" were the manager and report could both designate some co-workers to review the person, which was of course trivial to game for those who wanted).

There was also an implicit expectation of a bell curve. If you had an under-performing team, that'd be awesome news for at least some of the weaker members on your team (and conversely, I several times had reports who got short-changed because I was told too many of them were designated "above expectations" even though my manager and other people who had worked with them agreed, and so several of them were ranked lower to get the "correct" spread - it was absolute lunacy).

Unsurprisingly, come hiring time, I felt like we were under siege by an army of ineffective developers hoping to find refuge somewhere safe and hard-to-get-fired.

There are scarily many places like this for ineffective people to hide. Sometimes for many years.

I ran into a C programmer that did not get the relation between pointers and parameter passing. It was some scary awful code. Damn prep for code review almost got me in a car wreck.

Conversely, there have been plenty of people who I would call anti-workers. They technically accomplished "something" but the quality of work was so bad that the team would have been more productive without them. They were a net negative productivity for the team. At most companies I've worked at there was one of those.

For example I've been told that iterating over list of objects and aggregating (min/max/sum) some of their properties would involve introducing dependency on local installation of SQL server.

If that's the bar, then I'd estimate that roughly 80% of programmers in the valley right now can't step over it.

This debate is filled with so much heat -- and so little light -- because everyone carries around their own subjective definition of competence. I've personally never met anyone working in a company who can't code at all, but I've met plenty of supposedly elite programmers who are adrift without a framework. Are these people useless? I don't know, but I know they're interviewing other programmers.

If I think about it long enough, I go in circles and I end up back at the thesis of the article: this industry is filled with some spectacularly judgmental people.

If I had to reduce the debate to two clear groups, it's that one group believes that there's a bright-line distinction between "good" programmers and "bad" programmers, and that the questions-at-a-whiteboard model is an effective way of discriminating the groups. The other group doesn't believe in partitioning the world into two clear groups.

It isn't debatable that there are a lot of people who can't write code. I believe that trivial phone-screen coding problems are pretty effective at weeding those people out. Beyond that, it's much more about candidate/position fit, and that's a more subjective question. The problem is that programmers tend to be uncomfortable with subjectivity, preferring instead to posterize the shades of gray -- that's how we end up with the apocryphal Google engineer who can't code.

In your stated example (person hired; can't fathom basic realities of C pointers), I doubt that it reflects a complete inability to be functional in some programming role -- just not the one you were needing. Most "Rails programmers", for example, would probably be hopeless in that situation, but perfectly capable of cranking out websites with Rails. I've seen a fair number of real-world programmers whose (lack of) knowledge of algorithms and data structures makes me cringe -- and I certainly wouldn't hire them for the kind of work I tend to do -- but I have to accept that they're effective within their niche.

On the other hand: I think the counterintuitive thing about ineffective programmers is that many of them are perfectly capable of reasoning through a CS problem in Java or C++ or Python on a whiteboard. But they're --- for lack of a better term --- useless when it comes to actually getting real-world systems built.

My thoughts about why this happens aren't yet well-formed. I just want to contribute the perspective that the "bad programmer" problem isn't just about people who can't buzz a fizz.

Software engineering is a performative art. Sure, there is a heavy intellectual component, but you actually have to do actual software engineering to get good at it. You can't just study the theoretical aspects. It's like how you can't learn Haskell just be reading a book and some websites. And yet, someone could go through school for CS and slide by without actually doing any real engineering / programming. I had a CS graduate TA (for a CS discrete math class), at a very well regarded school, who literally did not know how to compile a C program, full stop. He was very into the pure-math side of CS...but still.

As an analogy - you can imagine that someone who has studied music theory for years, has perfect pitch, knows how all the various branches of music relate to each other, which artists influenced whom, etc...could still be shit at playing musical instruments or creating new music.

Recruiting is two problems: outreach and qualification. We did novel things on both fronts. For this thread, I just want to be pointing out: the changes we made to qualification were critical, instrumental, fundamental to our success. Most of our best hires could not have happened had we qualified candidates the way we did in 2009, and the way most firms do today.

It's only sometimes about people not being aware of a language feature. It's usually about people not being able to program.

   System.out.println("" + 3/4);

In reality you'd only use a mod operator if you were doing things like sorting into 4 columns or doing an operation on every 3rd thing. And it's not a concept that is introduced at school. Now I try to think of it, I think I only picked up mod when I was learning rounding in my first language and the man page happened to mention modulus at the same time.

And every dev should have hit the remainder operator at bare minimum when they had a long running loop and wanted to print a status every kth operation, eg

   for(int i=0; i < 100000000; i++){
     // some operation
     if(i % 100000 == 0)
       printf("** operating on count %d\n", i);
   }

    z++;
    if(z>100000) {
        print x;
        z=0;
    }

A brief search on SO and lo and behold:

http://stackoverflow.com/questions/1504420/c-what-does-the-p...

Viewed 38,000 times. Every language probably has a similar question.

While only "older" (I'm 39) developers will be likely to have been in that situation, it took another decade after I moved onto hardware with FPU's before I worked on anything where we actually used floating point math.

Instead we'd be working with fixed point stored in integer. E.g. for financial systems, floating point is a nightmare. Working with fixed point to whatever number of decimal points our accounting department wanted (5-6 typically) for tax calculations and the like was preferred.

So there are large number of areas where people can have worked successfully for many years without ever using floating point.

Part of the issue here is that each of us has a different understanding of what it takes to get a "real-world system" built.

That's somewhat obvious because each of us has different "real worlds", so even within the same language/ecosystem there is a huge disparity in what skills are required to be genuinely successful in different teams/organisations/applications.

To put it in concrete terms, when I worked in banking, the main impediment to building successful "real-world" systems was getting clear, unambiguous requirements that could be translated into something implementable. Some of the best developers in that organisation were excellent at their job because they could take the incomplete and inconsistent desires of a banker, and turn that into a coherent and complete "world view" about the system they were required to build. We were primarily a Java shop, and they could produce an adequate application within the frameworks available to them, but I shudder to think what they would have said if you asked about "coupling and cohesion" or the java memory model, or how dynamic proxies work, or even how they should choose between making a field a short, int or long.

So, for my current (startup) organisation, I wouldn't hire those people - despite them being instrumental in producing some of the most satisfying and successful (ROI) systems I've been involved in. This team needs people who can do hard engineering, and if my former colleagues applied for a role here they'd end up looking like the proverbial "inept programmer" that we're debating.

This is a reflection on their skills, but it's also a sign that similarly named roles ("software developer") in different teams require and cultivate different skill sets.

Thanks to the Dunning Kruger effect, people who are incompetent do not realize they are incompetent. I remember reading a follow-up paper to this study (I wish I could find it again!) which concluded that when people have their mistaken beliefs pointed out to them, it actually reinforces that mistaken belief! This means that when people actually engage in debate, the two sides entrench and become more confident in their positions.

I have actually argued that this may be what leads large corporations to have a kind of "talent inversion". The people with the least talent have the most confidence and volunteer for the craziest projects. The projects fail, but due to politics the failure is hidden and spun into a success. The original developers begin to believe their failure was a real success and any debate to the contrary only strengthens their belief -- which fuels their confidence. This confidence allows them to be promoted where the cycle continues and multiplies.

This leaves you with a strange situation. Everyone will agree that incompetence is a big problem in the organization. Unfortunately, everyone will point to completely different people when asked who is incompetent.

Let's say that you want to improve the level of talent on the team. In other words, you want to hire only people who are more talented than the average person already on your team. I think this is not an unusual situation to be in. If talent is normally distributed on the team then half of the team is below average (I hope I got that right because I suck at statistics ;-) ). But thanks to "talent inversion", you may have a higher proportion of people in senior ranks that are in the "below average" section.

Now you can have a debate about the what kind of people to hire: If person A thinks that they are at the top of the talent scale, they can't expect candidates to be as talented as they are. Probably they think it is completely reasonable to hire people who are considerably less talented than they are. Person B may actually be at the top of the talent scale and may be arguing strenuously that candidates must be considerably more talented than Person A.

Clearly such a debate will have a lot of heat without a lot of light.

So people at a place like Raytheon who simply can't code (or don't like to code) end up doing something else that they are better suited for.

Early stage tech startups just don't have a bunch of ancillary positions like that. Founders and early hires had either better be great at code or great at biz dev.

Also it's worth mentioning that it's not just a tech/engineering phenomenon. It's very common for people in other professional fields (law, medicine, etc) to get a little ways into their career and realize that the job is not at all what they expected or wanted or are suited for, and there are alternate career paths there as well.

If you can't code, you're probably still useful for other stuff. Businesses love to take people they already know and find better roles for them. It's way better than firing badly utilized talent and having to interview for some other candidate with that skillset.

I wouldn't go so far as to say, "There are no incompetent employees," as there are definitely worthless people in the workforce. But it's not as clear as a lot of people might think.

All these companies require degrees, check GPA, and typically have managers that came from technical backgrounds. While not perfect, this combination reduces the chances that a total dunce slips through. Also, all of the above companies tend to be looking at engineers/physics folk over cs/IT, although in the past 15-20 years a lot more CS.

I work at one of the above and for a long time only EEs were hired for any software positions. In regards to math ability that's a fairly high bar. And math competency is pretty well correlated with the ability to program at all.

Not saying that someone couldn't slip through. But all of the mentioned factors tend to reduce the pool of potential flops vs some of the now-standard hiring practices (from the article).

Thing is, I'm a decent programmer, maybe even above average. My career so far has consisted largely of being the only or one of a few technical people and doing whatever needs to be done to make X happen. I built a poor-man's version of the mail sorting machines the USPS (and other postal organizations) use to photograph, OCR and sort mail. I remember everyone's mind's being blown when Outbox (R.I.P.) showed people pictures of their letters, and I built a system like that by myself for a previous employer in about two years while also completely rewriting the website. 65kLOC of VBScript/ASP to 20kLOC of python/django plus a whole bunch of new features.

If you believe sloccount (I don't really) in two years I did about 4 person-years of work that it says is worth a substantial fraction of a million dollars and a substantial multiple of what I got paid.

But I don't remember what a red/black tree is.

Knowing data structures and algorithms isn't the only way to be a competent programmer. To get my EE degree I had to take calc 1,2,3, diffeq, discrete math, linear algebra, senior level statistics, and these were all just pre-reqs to my engineering classes.

I think a lot of people get hung up on computer science == programming. There's a lot of overlap, sure, but huge amounts of programming doesn't have anything to do with computer science.

The reality is different industries have different standards. I'd probably never pass a SV interview because I'm not ever going to memorize many of the CS fundamentals. I work with EEs, Aeros, MEs that code (robotics types), math physics and chem PHds, etc. Sure there's a few CS guys here and there usually that double majored in phys or math.

On a given day, what's more important to our engineering problems is my understanding of advanced linear algebra topics, not a sorting alg (lot of work in CV recently for ex). We really don't bring anyone into an interview that hasnt studied the full gamut of math required in a math/phys/engineering degree. In such an environment, the interview questions are more about how you solved problems and never the memorization of some sorting alg. Oddly enough, when you weed out everyone that doesn't have a full math curriculum first, you don't seem to find many people that 'cant code'. Go figure.

How you solved a problem is probably the only meaningful way to screen candidates. Sadly there are a lot of people who think that a whiteboard coding session is "how you solved a problem" when few -- if any -- meaningful problems get solved in an hour or two.

When I get really stumped on something the first thing I do is get my head out of the problem. Go for a walk, work with my hands in the shop, get lunch, anything but try and solve the problem directly anymore. This strategy used to make me really nervous in college with short deadlines but now that's less of a concern.

I have yet to step back from a problem and not figure out the solution (at least where one exists).

Another part is the inherent error is using education as a proxy for ability to contribute.

My biggest concern, however, is that we assume there is a hierarchy of programmers. That the person who surpasses John Carmack is automatically more valuable than someone else, because it's a vertical pecking order.

In reality, it's likely that given the constraints of a position, Carmack++ is not a good fit.

We see someone build a faster sqrt algorithm as ability. What about the programmer who does his best to navigate internal obstacles adroitly -- why is that not praised just as much? It's certainly a difficult accomplishment -- why praise people for ability entirely?

I have a friend who is a mid-level manager at one of the listed companies, who came in without a degree, and was promoted because of proven competence to where he is now, and is spending several hours a week working on his degree because he's been promoted to the point, none of his superiors will let him be promoted further without a degree, despite handling the job better than his peers.

On the flip side, I've worked with people with a degree in CS (or similar), who couldn't develop a good software system if their life depended on it. The other side of this is, that many people don't continue to learn and only stagnate once they become familiar with one system. Software development tools don't stop coming out... there will always be new languages and platforms, but I have always seen a lot of resistance... I'm one of the few 40+ y.o. developers I know who keeps an eye on what's coming up in the community.

But my point was about Statistics. The HR warlords for all those defense/aero companies that have set these limits have done so because it segments the pool of candidates to one with a lower chance of bust (or so they've decided as such).

I'd like to see data, I'm some HR or consulting group somewhere has it. Or maybe it doesnt exist. A simple logistic regression problem with a high number of variables (degree or not, age, major, self-study, continuing ed, etc etc).

edit: also - all my comments are speaking from the engineering side of these companies. not IT. So the actual work content is a bit different than the original article may be concerned with.