Maybe not with today's technology, but in the future, they will still have the recording and will have the technology.
It's the same thing with encryption. It's safe today, but in the future it'll be decodeable, and they'll still have the encrypted copies to work on.
They can eventually know who I am by voice recognition, especially they can get corroboration by the fact that we live in the same building.
I imagine we probably said something that would suggest I live nearby (... "Checking database" ...) or that I'm a regular visitor (... "Checking database." ...).
They could be sending an inaudible tone, to be captured by an app on my phone. I don't have an Amazon app, but some other app might capture the tone and sell that fact to Amazon, so that they don't leave money on the table.
I was visiting my neighbor the other day. We were wondering something about special counsel mueller, and she decided to ask Alexa. This was new, and I was surprised she had this surveillance device, but I didn't say anything.
There was some back and forth, then I said to her "I think you're ordering motor oil "
"Alexa, stop! Alexa, stop!" As if a dog was chewing on the furniture.
And now Amazon knows that I was discussing politics, and my opinion. That kind of analyzed data is going to be lucrative in some future military or law enforcement or political contract.
Rule 1. Don't leave money on the table.
Rule 2. Ever.