Sure you do. At least if you're actually trying to understand what the words mean in the context they're used. I understand your "whatever floats your boat" example, including the cultural context. This tea business is completely foreign to me though.

I have no problem understanding these things. It's easy to infer meaning when they're used in actual conversation. When taken in isolation and without cultural understanding, these idioms sound like those made up sentences used for language practice.