One major issue that you're glossing over is that Alexa is easily triggered, even when no one explicitly says "Alexa". So audio clips of private conversations are being sent to Amazon.

For an example of accidental triggering, look up news on "Alexa creepy laughter".