I'm not sure whether it's listening to more than the wake word, but I've seen Siri wake up quite often when I very definitely haven't said anything approaching "Siri", and see it occasionally on other people's devices too. I remember listening to a BBC podcast in the car once and there was once piece of audio from it that would reliably activate Siri. I was a bit nonplussed by it and rewound it four or five times to check, which it did every time. I think accidental activation is a much more likely explanation, which is still dreadful from a privacy perspective.