I also don't want to shop around for roads, police, fire service, garbage collection, or any number of other things provided by my taxes. I definitely don't want people less well off to go without those things because they couldn't pay. The point of taxes, and particularly progressive taxes, is to ensure everyone has access to services while placing the burden for that on those most able to afford it. That isn't particularly well implemented in many cases, but I strongly believe it should be the goal.
Does it necessarily? Corporations are looking for profit growth, and the customer will pay for that.
Imagine a service where a public provider is replaced with three private corporations. Superficially there's competition and freedom of choice... But what if the three companies all operate at 80% margin, are constantly squeezing on quality to drive the margin even higher, and they are owned by multinationals that spend enormous amounts on lobbying to prevent new entrants in the market? It's easy to see how both price and quality can be worse in this case than it was with the single public provider.
Government are ultimate monopolies: always getting worse and more expensive.
Maybe you are thinking about a handful of functioning, high quality governments (US, Germany, Nordic States) but I am thinking about the great majority of the governments on the globe (Africa, Asia, E Europe) which provide really crappy, expensive services.