Well my observation is that programming puzzles that are top in the difficulty tree often typically involve trees, graphs, queues, or bit manipulation and in my experience, they aren't usually encountered in programmer's average life. As it is not easy to get them right the first time for an intense visualization exercise they involve, preople often find them lost and give up their hands.

I'm a student so please don't think I'm trying to be above you here, but could you clarify this a bit? Maybe I'm getting suckered into the academia way, but I find it hard to believe most programmers will never need trees, graphs or queues?

Programming is very vast field and each kind has different requirements and challenges. In programming kernel, compiler or any complex software, the need of data structures is inevitable but in web development, you rarely need them and I can safely assume even frameworks build on high level language like Ruby or python, don't need them at all. Of course, there apps like Akinator which are an exception.

Some people believe that how web development is a cakewalk, in comparison. While it might be true to a certain extent since developers are cut off from challenges at lower level but there are difficulties that can take toll on you, trying to work with huge interactive JS application is no piece of cake, for instance.

Most people aren't equipped to use trees, graphs, or queues so they don't use them. (Well, that's not so true of queues...) Meanwhile it is true that a ton of the interesting things done involves trees and graphs of some sort or another.

The problem is that abstractions are leaky. E.g. when I first started learning C++ in high school, I didn't know a tree if it bit me. But, the STL has a set implementation. If you take a closer look at the docs, you will find out that it is implemented as a red-black tree. I am not sure the assertion that you will never come across a tree or a graph in your life will be true always.

There's nothing that says std::map has to be implemented as a red-black tree. And using an abstract data type that's implemented with a tree is not the same thing as making your own tree for your own purposes.

> There's nothing that says std::map has to be implemented as a red-black tree.

Of course not, but understanding what that means is key to using it. If you think a red black tree is some arcane bullshit that you never are going to use, you will probably skip over it when you read the docs and get bitten.

> Of course not, but understanding what that means is key to using it.

Not at all. A user could think it's a hash table and they'd still be likely to maximize the potential utility of the type in their application. They could understand that it's implemented with a balanced search tree without knowing about the details of the balancing algorithm and that's pretty much the maximum utility right there. (In fact it's a good mnemonic for understanding almost all the performance and iterator guarantees of a std::map's functions.)

And you're not going to find that it's implemented as a red black tree in "the docs". You generally don't know that. All you know and all you'll see documented is the performance requirements and iterator invalidation behavior of various functions. I mean look at the docs. That's what's there.

The real benefit of being comfortable with trees and graphs is so that you're comfortable with actually using trees and graphs yourself, directly, when they'd be useful to manipulate. If you want to look at a commit log, well, that's a graph, and I guess you can't work with that if you claim that graphs are weird obscure stuff and cringe at the word "acyclic". Want to use a filesystem? Apparently that's a hard thing for some people, sadly enough because it's a tree (or graph, depending on your attitude about soft and hard links). That's where you're writing code that manipulates the actual data structure, instead of just using some lookup-table interface.

> The real benefit of being comfortable with trees and graphs is so that you're comfortable with actually using trees and graphs yourself, directly, when they'd be useful to manipulate. If you want to look at a commit log, well, that's a graph, and I guess you can't work with that if you claim that graphs are weird obscure stuff and cringe at the word "acyclic". Want to use a filesystem? Apparently that's a hard thing for some people, sadly enough because it's a tree (or graph, depending on your attitude about soft and hard links). That's where you're writing code that manipulates the actual data structure, instead of just using some lookup-table interface.

I am not clear how you are disagreeing with me here. :) Surely manipulating commit logs, understanding say how Git works, squashing commits, rebasing, manipulating access to the file system, surely, these are things that are not out of the realm of being asked to do. Hell, if you are dealing with a freaking linked list at some point or not in your career, you are dealing with a DAG.

Frameworks and open source handle the heavy lifting. Some coders aren't interested in learning how a framework/function/class works, they just know the end result.

> why do you say you can't solve puzzle problems? How many have you tried?

Frankly speaking, they do not interest me.I would rather collaborate with friends and work on a project than solve competitive problems or puzzles.

Couple of questions (again following up on OP's hypothetical stream).

1. How is a small team of programmers to know that you will be able to collaborate with them effectively within the space of an interview ?

2. What exactly do you mean by "competitive problems" or "puzzles"? What is off-limits?

I am asking 2. to identify what you mean by a puzzle. For example, let me take the substring matching problem. There are a ton of ways of doing those which for me traverse the realm of being both puzzles and also in some ways important computer science questions [1]. You could do brute force. You could have a moving hash window and use that to check if the substring is present. (Sort of Rabin Karp). You could build a trie called a suffix tree. Now the way intuition works (at least for me) is that you either know how to solve it, and then you sort of recollect the details in a few minutes. Or you have no idea of how to do it and probably don't know how to create a solution like that unless it was super critical that you go figure this out. And the problem is that the real world never throws things straightforward at you. Curveballs at least in the engineering world require transforming previous solutions into something new. So unless you grok trees, how will I know if you really will be able to spot that a random problem is just a tree problem with some modifications and then go ahead and implement it, instead of say spending a few weeks refreshing graphs.

[1] http://cstheory.stackexchange.com/questions/19759/core-algor...

> How is a small team of programmers to know that you will be able to collaborate with them effectively within the space of an interview ?

Isn't my github profile enough proof to show that I can collaborate with people? Also, as I pointed out in my recent comment: Eran Hammer, (creator of OAuth and Hawk) presents a new way to hire engineers. He suggests the concept of a 'Homework' wherein prospective employees are presented with a problem/bug that they are actually facing in their product. One of my acquaintance from Mozilla also pointed out that he was hired in a similar way.

> 2. What exactly do you mean by "competitive problems" or "puzzles"? What is off-limits?

For me, competitive problems are the kind problems for which interviewstreet/codechef etc are known for. At times, they involve the kind of problems which a developer is not likely to face during his time at the company. For example, being a web developer, I don't think that there will be a time when I need to write a string matching program from scratch. Wouldn't questions related to scalability/databases/some selected data structures make more sense for such a profile?

Don't take me wrong, I am not against questions involving string matching/sorting/kernel concepts, all I am saying is that they should be asked from those who are really going to face such problems while working.

[1] https://news.ycombinator.com/item?id=6828411

> Isn't my github profile enough proof to show that I can collaborate with people?

It doesn't show if you can collaborate with these people. Which is one of the things that I look for when I interview folks or interview at companies.

> For me, ... while working.

So I have this opinion that most jobs don't require even a C.S. degree or even knowing more about programming than cobbling together a few libraries and getting something out. Now, most companies don't agree. You said you learn fast right? This is a lesson you should start learning then. Most companies look for some mythical beast called a "10x" developer. They have no idea how they can solve it. Sure, they can test you on the problems they are working on right now. However, they have no idea what problems they may face in the future. So they test you on a bunch of random "hard shit". You jump enough hoops, you get hired.

/cynic