I'd liken our jobs to those of pilots. Most of the time, anyone can fly the plane: just punch in the flight plan to autopilot, and press 'A/P engage'. However, when a pilot is near cumulonimbus clouds with an engine failure, he is forced to step up to the plate. This is what pilots are paid for: to handle the crazy emergency situations that can arise when you're in the air. This, too, is what good engineers are paid for: handling the difficult problems that occasionally arise during development. If you're not hiring for this, you're hiring code monkeys.
In my case, I had to show design capabilities in addition to code and being creative didn't mean just applying hacks found on the internet. But my boss had to actually see if I can get through with building the project using a combination of existing knowledge and the ability to find solutions.
My challenge was to build a discussion forum from scratch with user registration and a reasonably attractive UI. So that's PHP + Photoshop + CSS/HTML/JS
3 Tables "posts", "replies", "users" (replies was a closure table), topics index, topic view page, a form page for registration and one for the profile page. JS form validation and a simple layout via CSS and HTML5.
It was more of a Twitter-lite (a very poor man's Twitter-lite)
I just hired someone for a job who's first response was to copy and paste the error code into Google and view a stack overflow post with a similar problem. From there he took what the response on stack overflow said and crafted a solid solution to the issue. It took him about an hour and I saw that he had to Google a few functions (Python language) but he showed he knew the resources he needed and the basics to coding solutions to problems.
I don't care if you know how to create a linked list in C on a piece of paper or why we have round manhole covers instead of square, this isn't useful in the real world, I want to see if you can solve a real problem and the method you go about it.