That might be an unfair characterization. But only just.
But really, if someone can describe the detailed psuedocode for an algorithm and explain its runtime characteristics, are you going to make them write out what they have just said?
Personally, I think the talking is a lot more valuable than the whiteboard. I rarely have to hand-write complete algorithm on a whiteboard, but I regularly need to communicate technical ideas accurately and succinctly. That's what I'm really trying to demonstrate anyways: of course I can code, but I can also organize my thoughts well and communicate them clearly to others.
Honestly, in my experience, most application problems can be reasonably talked through without writing any code.
I share your experience that most problems can be reasonably talked through without writing code, but most people here are talking about a specific kind of interview that does not work that way.
Just be honest (immediately so), be humble, and have confidence that you will do your very best. What ever happens after that you don't really control. It is okay to be nervous, but keep it in context. Nobody is shooting at you. You aren't drowning. Your housing isn't burning down. It is just a conversation.