I have had a problem with using the speech_recognition library in that it does not stop listening when silence occurs.

After trying to tweak the threshold parameters without success I just figured I'd add a custom key-command to break the listening loop in my project.