We use smartphones for the same reason we don't avoid any non-free software like Stallman: a huge amount of additional possibilities. But what Alexa, Siri etc. offer is just a different way to use a limited set of existing features, in exchange for a much higher risk of a privacy breach.

Everyone needs to draw a line somewhere, I just don't see a reason to use voice assistants, definitely not with internet connectivity and where the company keeps all recordings for some reason.