There are actually a few projects that follow this model much closer than Alexa/Siri/Google Assistant, or at the very least let you control host the computation on your desktop so you know where all your traffic is going.

https://mycroft.ai/ https://snips.ai/