Correct me if I'm wrong, but a reasonable approach would be to give the interviewer the benefit of the doubt and to at least engage the programming challenge, and to also voice your concern with the process. It's no different from voicing your concerns first and grudgingly engaging the problem, but the two make wildly different impressions.

Also, as an interviewee, you stand to learn a lot more from that interaction than by rejecting it. Hell, I welcome coding challenges as an opportunity to size up the interviewer's chops!

If the challenge is conducted in a reasonable way, it should feel like any other design/pair programming session you've had at the office, and it can be quite revealing of the tendencies of your potential colleagues.