That's not really fair, people don't grow unless they're challenged.

Some people really do never advance past a certain point, but a lot of people write someone off as mediocre when they're just still in the process of gaining skill. Also, they only assess ONCE, which is pretty bad if we're trying to establish how good you are forever and always.

That's true, but how do you discern between someone who has potential and has not yet been challenged, and someone who simply doesn't care? In my experience, the best will challenge themselves to learn new things on their own.

to me, complex, and complex-ITY are entirely different matters. I want a smart programmer who can figure out really complex bugs (something you cant figure out from google/wikipedia). Not someone who memorized the big-O performance tables of 8 different data structures (something you CAN look up on wikipedia ).

Exactly. I once identified memory leaks in a managed runtime implementation, worked around them, and showed that third party networking libraries from the hardware vendor were causing irrecoverable crashes but I can't spout off runtime complexity of algos. The former saved a multi million dollar contract and my company's reputation, the latter is something I look up when I need it.

Optimally, I would like both. I agree that wrote memorization is not a very useful skill, but I'd like to know that they can grasp concepts, and I find it hard to believe that there are a bunch of skilled devs out there with great potential who don't or can't understand algorithmic complexity. Interviewing is hard though, I'm not trying to say that any one question or metric should necessarily rule someone out.