Would depend on my budget (because it would certainly cost more than hovering up all training data for free).

Perhaps: Pay/reward for data sharing, decentralized training, differential privacy, local training and submitting the weights, local fine-tuning of pre-trained model, marketplace of third-party (open source) models, ...

... or just have the user repeat some phrases for 5-15 minutes during initial setup? Like you could do with good ol' Microsoft SpeechAPI from the first decade of this century?

The obsession with cloud-based machine learning is creating half of these problems.