I hated Chocolatey. Those binary shims seemed to produce slow, unstable binaries. There also still isn't a fully-usable Chocolatey backend for the PowerShell package manager.
The Programs & Features pane works just fine for me as a package manager.
Also, take a look at Chocolatey for package management. Yeah it's not a built-in thing, but it's pretty decent.