First, the markup in just sheer ops terms is enormous. Minimum 4x, and in some cases 20x. Sure, you get a couple more nines of reliability, but 99% of startups don't need that much reliability.
Second, in countries like Canada where capital expenditure on computer systems can be heavily marked down[0] people still use AWS and DigitalOcean by default.
The main reason people don't do colocating is that it is a waste of time better spent. DigitalOcean is cheap enough. AWS is cheap enough and if the team doesn't have experience running Postgres, fine, they use RDS. Sometimes it's just sheer laziness at first that later blossomed into "holy shit, we're growing too fast to even think about migrating off of these giant AWS bills" which was essentially Dropbox's story if I recall correctly.
[0] 55% per year, and in practice they allow full expensing in the first year for petty amounts of outlay. For example, if your business is a data centre, sure it's 55% per year, but if you run a flower farm and fully expense a new laptop for sales the CRA is generally fine with it.
(Of course you have to force your devs to do the ops stuff now, but that's not accounting and HR's problem now.)
It exists to shift CapEx to OpEx for accounting. Any technical considerations are a distant third place on the list.