But, on the other hand, if you apply to a company, especially one that's more or less famous, puzzles are a very good way of separating the wheat from the chaff on a level playing field. Evaluating a strangers performance based on their solution to a common puzzle is very time efficient and allows the company to process large numbers of applicants fairly.

A common notion here is that great people can just pick up the phone, or will be actively courted by companies, so they don't need puzzles and recruiters. That is great for them (honestly, not a snark), but it's not scalable: Not everyone can know everyone, and there has to be a way for newcomers to enter into the network.

Case in point: Stripe - you want to work for them, but apparently you don't know anyone on the inside that can recommend you. And apparently you're not on their radar already. They do great stuff, they seem like an amazing bunch - so there are likely to be hundreds who want to work for them - and most of them, regrettably, are not qualified enough.

Do you really think it's fair to require Stripe to assign a member of their engineering team (that means you after they hire you) to review every strangers GitHub accounts, when you refuse to invest even a little time in signalling to them that you're the real deal (the puzzles aren't all multi-day endeavours)?

Yes.

Investing time in open source software is a better use of time from the point of view of a candidate, as they can show the same code to many different companies.

So it comes down to who is more in demand. Is your company so desirable that good programmers will invest a significant chunk of time just to have a chance with you? If the answer isn't an automatic 'Yes', maybe you should rethink your hiring strategy.

It also doesn't seem particularly hard to automatically filter out candidates with no GitHub account, or a GitHub account but few contributions.

The Github filter is a bad one, though - lots of good programmers don't do the whole open source thing, and we need to stop this pretense of a good Github account being necessary for hiring. What started as a suggestion in some hiring blog posts has become overblown into dogma now.

Artists maintain a portfolio of their work; why shouldn't programmers?

Sometimes it's best to evaluate the work rather than what goes into it. Granted, this isn't always the case and can't be attributed to multi team environments, but I've often been asked for code when I have a large number of (successful) projects solely developed by me.

Also, small code samples are fine but some employers are in the habit of asking for major work (demos) just to show off your coding chops. That's bad.

For a developer, the how matters as much, or even more so than the result. You could create a complete mess that still manages to just about work, and it wouldn't be obvious from a cursory execution of the binary just how bad it was.

Ideally, if I was going to hire someone, I'd want to know how they develop a reasonably-sized project. An interview or puzzle can give me some idea, but a GitHub account is far superior because it would show me how they work over a longer time frame, and perhaps with other people. I honestly can't think of anything that comes close to demonstrating skill.

So the fact that you can actually see more for a programmer is actually a bonus, but that doesn't mean it should be abused like plenty of companies try and do these days.

Risk aversion is fine, and obviously you want to hire the best candidate but like mentioned elsewhere on this thread:

1. Not everyone has public github projects and I hardly believe anyone should have to work on open source stuff just to show off their coding skills.. Thats really not the right reason to work on open source stuff.

2. For web projects while you might not have access to backend code you can always look at the front end and see if the guy is any good. This is especially true for me considering the fact that I have multiple projects developed from start to finish live and available on the Internet.

Again, small demos or code sample requests are always fine. However, an entire project to get a job somewhere ? Might have been okay when I had nothing to show for it, but after a bit of experience on your resume, it gets old..

There's no such thing as fairness in the job market. If Stripe can't find enough programmers to fill their needs, then yes, they need to allocate their own labor to recruiting. At some point, the business cycle will swing the other way, and it will become necessary for programmers to put our time into hunting for companies who want to hire even one programmer.

C'est la vie.

... then allocating the programmers they have to sifting through a completely open "jobs" mailbox is still probably not a good idea.

> At some point, the business cycle will swing the other way, and it will become necessary for programmers to put our time into hunting for companies who want to hire even one programmer.

For the level of programmer competence we're talking about here, that hasn't happened yet.

Define that level non-tautologically then, please.

Wouldn't that still be better than working on some contrived game that has no real-world relevance for free?

It still seems like a positive step. No quizzes at all seems like an even better way, but I did qualify the point with the assumption that the quiz would happen regardless.

Not to get too meta, but what if the company essentially buys your argument? "Yes, this is a non-real world problem for entertainment purposes only..."

But they want to find programmers who find such problems entertaining.

The problem with puzzles is just that. You ask questions and hire candidates based on issues that have nothing related to the job. The end result is you up hiring the wrong people and then say you can't find good programmers.

You can't find good programmers because you are not asking questions meant to hire good programmers. You wanted puzzle experts and that is what you got.

http://itasoftware.com/careers/work-at-ita/hiring-puzzles.ht...

These aren't classic Microsoft "puzzle questions". They are programming challenges that say a lot about your approach to programs and problems. In particular, all of them ask you to make tradeoffs between correctness and performance.

They are cool enough to attract engineers, hard enough to discourage the idly curious, and unique enough that no published solution exists. That's why I said that someone who does this kind of puzzle for the sheer enjoyment of it might be exactly what a high-level software shop is looking for.

Would I want a "puzzle expert" who could solve the problems on ITA's page? Hell yes.

Granted, it is not a thing you should give everyone. I can usually tell when I look at a resume that such a thing is likely to be pretty pointless (It points me to a github account I can look at instead? Why bother? The resume is clearly written by someone who knows what they're talking about and isn't stuffing everything they've ever done onto a page in a game of buzzword bingo? Probably safe.) so I don't give fizzbuzz tests to everyone but sometimes you have to be sure.

Hopefully I'll have some good github stuff to point potential employers to soon (i'm working on some things now). But at least I know that I should be able to pass some of the incompetency gauges fairly easily!

also, i've never heard of buzzword bingo before, but it sounds interesting! maybe i'll make a program for buzzword bingo. You can use it in your interviews! :P

But actually, fizzbuzz is just the bottom level. You also want to do basic coding exercises for whatever specific area you are hiring for, such as:

- Set up an extremely basic web app that lets the user enter a name and echoes it back to them.

- Make a sprite move across the screen.

- Display a 3d cube with different colors on each side and make it rotate.

- Make it play some music.

- Aggregate the data in this simple database.

- Here's some naive producer-consumer code. Make it thread-safe.

- Debug this code that has memory issues.

- Critique this (flawed) design.

- Update the CSS to make this page look nicer.

Basically, you never want to get into a situation where you've hired someone who has managed to talk his way through life. It's far less headache to just use a few sanity checks on every potential hire.

  sub isIP {
    return (shift =~ m/^((\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])\.){3}(\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])$/)? 1 : 0;
  }

The Stanford CS Ph.D. candidate whose solution included a Perl function for converting decimal numbers of up to three digits to binary was rejected.

- writing a subroutine

- understanding Perl parameter passing

- understanding Perl regexps, including =~, anchoring, alternatives, metacharacter escaping, character classes, and repeat counts;

- the ternary operator.

And then you could probe things like:

- Do they know how to fix the bug in your version where it incorrectly accepts an IP address with a newline appended to it?

- Do they know it would still work without the return and the ?1:0?

- Can they figure out that they could simplify and clarify the code substantially with qr?

- Do they understand why someone might prefer a non-regexp-based approach? :)

The guy was writing some really weird stuff. When asked to explain, he confidently told us that in C++, every time you reference the iterator variable inside the loop, it gets increased. His solution was something like:

    for (int i = 1; i <= 10; i++)
    {
        printf("%d\n", i);   // <== i gets incremented!
        i--;  // need to compensate for the extra increment!
    }

So, yeah, I see FizzBuzz as a wonderful way to weed out the incompetent right away, allowing you to focus on deciding whether you need capable or exceptional.

Incidentally, one of the guys on my senior project team in college didn't understand how variable assignment worked. He would write:

    int flagsVar = COMMON_BASE_FLAGS;
    flagsVar = ADDITIONAL_FLAG_I_WANT;
    CallAPI(flagsVar);  // Why doesn't it work?!