In theory, no. A private company (by the commonly understood definition) always needs growth and profit to continue providing a good service. Neither of those things are compatible with being cheaper than a provider that doesn't have to optimise for those things.

Obviously in practice that's not how things work out, so in most cases private companies end up moving faster, leveraging efficiencies, utilising innovation better, and generally being cheaper. The lesson shouldn't be that means the government shouldn't provide services though, but rather that the government should act more like a business and do things businesses do to be competitive.

It's very noticeable that when government does this it works really well. The UK government has brought a lot of it's web and IT services in-house in the past decade, and they're so much better now.