I personally don’t have a problem either way, but there is a nuanced difference.

If someone presses the home button to actively activate Siri they are purposefully sending messages to Apple and shouldn’t be surprised that the data is being processed and maybe even listened to by humans.

However, I could see someone being slightly more concerned about accidental activations causing data to be sent to Apple.

It would also be easy for Apple not to allow humans to listen to recordings triggered by Hey Siri.