Sometimes it's not. But if you're determined enough to spy on somebody via the audio on their phone, you will likely be persistent and eventually overhear something the person would prefer you didn't. If I were concerned about being spied on I wouldn't dismiss it so easily.

Although you are correct about the pocket thing, the point is that you don't know when someone may be listening to you.