Don't search around for headphones, just put your phone to your ear and it will play as if a phone call, not the speaker phone.

Also, WhatsApp and Signal allow you to play the voice note at faster speeds, like 1.5x.