Both where that line is, and how they handle the failure tells you a lot.
(Caveat: Make sure you're keeping it on the fail/no-fail edge, and you treat the candidate with respect. The point is not to show them that they're wrong, but to explore the boundaries of their skill set. Without being mean is a major goal here)
And if you can't push the candidate there: HIRE THEM. Right away. If they know that much more than you do that you can't get them beyond their limits, they are a fantastic addition to your team. (Provided they match personality-wise, too)
As most of a job is about doing things that haven't been done before, getting the candidate into unknown territory is when you really find out how effective they will be. Some people just stop, others switch seamlessly into problem solving mode and come up with an answer that may or may not work, and also with a plan to test whether it does.
A candidate and interviewer both have to be fairly relaxed and comfortable to do it properly, but that's more of a problem for people new to the interviewing process (probably not senior management roles)
Some companies do this intentionally without hiring anybody in order to double-check their plans. It's basically free consulting.
For people hiring, go ahead and use this as a first-pass filter, but realize that it's easy to game (by someone who's trying hard and learning, so there is a positive signal in that). It's going to be important to go through and work with them for a period of time in a let's-date-before-we-marry relationship. As an employee, I always insisted on a contracting relationship before an employment relationship.
Like I said in the footer, Heroku really seems to do this the right way, and it shows. Kudos to them.
And everyone else needs to step up their game.