That makes me terribly uncomfortable because Microsoft is likely to just silently turn it back on in the future.

I used Powershell scripts to uninstall a bunch of the bloatware (XBox Music, Money, etc...) and they just come back. It's very frustrating.