You are right, “always listening" they mean it's sitting in a low power state listening for a very specific trigger like "Hey Siri" or "OK Google" and literally nothing else.