The Pebble Core doesn't have a built-in microphone or built-in speaker. You need to either plug in a headset with a microphone or use a Bluetooth Headset with microphone. If you used a wired headset with no mic, it can't listen to you.