Usually they don't really understand what I mean, and I just elaborate by explaining that I have no interest in working for a company that uses this type of brainteaser to decide who to hire.

One time the guy got half angry, started defending his process and just walked me out. Another time they tried to move on to the next interviewer and I just said I wasn't interested anymore, it would be a waste of time.

Obviously I never got a positive response after saying that. I still dream of the day when the interviewer will shout "that's right! That was the test! You passed!" :)