In practice, it almost never works out very well. Puzzles are incompletely specified, impossible to answer unless you guess at assumptions that are sure to be wrong, or they take so much time to solve it's just not worth it.
Usually companies don't need complex puzzles for screening anyway. Something rather trivial and quick to answer like the fizz buzz problem eliminates 90% of the dud applicants anyway.
Designing a good problem is hard. Something that's difficult enough to present a challenge but easy enough to be solvable by a good programmer in ~15 minutes or so takes some thought. But this is another reason why I appreciate a good programming problem in a posting.
If you do a decent job of publicizing an opening you'll can easily get 1000 resumes. If you narrow this down to what seem to be the top 20, 18 of those won't be able to write a simple loop in the language of their choice. Part of the reason for this phenomenon is that the best developers who applied were in the 980 resumes discarded (this is in part due to unreasonable expectations on the hiring side and outright fabrication on the applying side).
But the general idea is that the same duds apply to every job. If you post a job ad for a terrible job with unreasonable parameters, then 100% of the applicants will be the duds as everyone with any sense reasonably ignores the ad. The more reasonable and realistic the ad, the more potentially qualified candidates will apply, but this number is not going to be the majority of applicants. The majority will be the non-english speakers and DeVry Institute grads who are mass spamming every job ad they can find with bogus resumes.
There's no point to starting with a challenging programming test because the interviewer's idea of obvious common sense insights is typically some highly domain specific thing that only is known at the particular company. Or even worse, the "challenge project" which will take a week to finish and is obviously a real work project for the company that they want done for free. Furthermore, interviewers seldom have any realistic idea of what could really be solved in 10 minutes by someone who hasn't seen the problem before like the person who selected it. FizzBuzz like problems are again the answer. Ask to reverse a string without calling a library function. This is a reasonable 10 minute question. If 95% of applicants fail, then this is a very useful screen is it not.
You can give your candidates this problem during an interview, but only if you are willing to give up time you could have otherwise spent in that interview getting to know the candidate in other ways.
These kinds of puzzles communicate arrogance to me. They say "We're more interested in excluding you based on a trick question than in getting to know you".
For example, the puzzle in this ad is a good one: http://newyork.craigslist.org/mnh/sof/1927765905.html
Resolving such ambiguities and unspecified detail is something we deal with every day; however, it's less than helpful so early in the process. As others have mentioned, hiring is a two way street.
Reading in the graph and putting it in a tree structure is a reasonable problem, but not a 10 minute interview one.
Traversing the graph to find the optimal path? Wow, that's known to be a tricky problem. Brute force search will work but will take a while. Now, undoubtedly there must be some very clever proof that discards most of the searches for the special case of a triangle structure. And if you've seen that you're all set. If not, hopefully you have your copy of AoCP at hand to start looking for it. If you are in an interview and haven't seen it, you just need to recreate the conditions from 1 million BC to 1950 or whenever someone had the right insight to come up with a proof. Not really a reasonable assumption. In those circumstances it's only a test of if you've run into this issue before.
And figuring out the answer is necessary to determine the email to submit a resume? How foolish that hiring manager is. If he was Google, yeah, you can put the question on a giant billboard ad as a challenge and get some responses. He's not Google.
Reasonable question is if someone can parse some data and put it in a tree or graph.
Reasonable interview question is to bring this up and talk about approaches. Person mention's Djikstra's algorithm, well he's heard of Djikstra and knew he was big on optimal path searches. Mentions AoCP, he knows a good place to look for clever algorithms. Knows how to solve it on the spot? Probably pretty clever. Finding the optimal 100 node long path in a graph as a screening method to send in a resume? Funny stuff there.
Being a bit clever myself though I'll go a bit further here. I'd say you might be the person who posted that ad. But I think more likely is you want to apply for the job and have no idea at all how to solve that problem and are hoping by coming here and pretending it is trivial someone like me will give you some hints. Because if you really knew your stuff, you wouldn't be claiming it was trivial for any competent practitioner to solve and a reasonable screening question.
Sure, there is an easy answer out there for doing this search, that's almost certain. That everyone competent would figure it out easily? Nah.
Traversing the graph to find the optimal path? Wow, that's known to be a tricky problem.
This question wouldn't be appropriate for an interview, but I think it's just right for a pre-screen. It's not hard if you realize they're not asking for the optimal path, just the biggest total.
It's not my question, but I did take a second look at this company and I probably wouldn't have based on the rest of the posting.
The Quora challenge problem, on the other hand, seems like very hard one: http://www.quora.com/challenges
They aren't asking for a good enough path, but for the best possible path, the one that gives the biggest total. That's the optimal path for this problem.
Probably a brute force search is the way to go, I certainly can't think of any way to thin this problem out without doing research, especially given the requirement that the optimal (maximum value) solution is required.
99 levels below the top, each time a binary decision, there's 2^99 = 6.34 * 10^29 paths only, won't take too long relative to the life of the universe since 2^99 nanoseconds is only 20 trillion years. That's the answer, might have a little problem with the sun burning out before the computation completes, but by that time you should be able to relocate using a warp drive. Good luck on the interview.
edit: Ah I just figured out the pruning algorithm that will solve it. And that's the answer too, I just gave it to you in the previous sentence if you read carefully. Obvious once you think of it, not until then. I doubt many people would figure it out without a hint. And with a hint only clever ones could get it. As a screening to send in resumes? Pretty useless. If you do get a response who's to say they didn't ask for hints on a message board or pay someone to solve it on rentacoder.
(Cue: reactions from others that it's a one liner in Haskell, or that it's obviously simple and easy since the solution was so short.)
(The five lines didn't include the data, but a sixth very long line takes care of that if really felt to be part of the count.)
1. this wasn't posted by a recruiter or somebody clueless in h.r.
2. engineers at this company still care about hiring
3. a lot of buzzword bombing b.s. artists aren't going to be applying
4. the company recognizes that raw aptitude is more important than bullet list bingo
In other words, it's a sign that the engineering culture is probably still healthy at this company.