Not to mention compliance. If you have to go through third-party audits (and most businesses do after reaching a certain size), then using standard services from a public cloud provider can simplify things significantly. Sometimes that's because you can just offload "security of the cloud" to the cloud provider, and other times it's because there are well-established guidelines for how to achieve, say, PCI compliance on AWS.