The axe you're grinding is profoundly weird, and indeed a large part of my business is because the stuff we build is extremely cost-competitive with dedicated hardware. Difference being that I can open up the console and start shooting servers and nothing breaks. You're not saying the same with the overwhelming majority of naive "dedicated hardware" deploys, especially at the levels of skill and expenditure that small companies can employ.
You're right, of course, about naive deployments. But it just isn't that hard to build reliable systems, assuming some experience. And if you're doing anything more interesting than pretty CRUD forms (say, atypical storage or bandwidth requirements), DYI becomes much cheaper, fast.
To reiterate, yes, you need someone who knows what they're doing on the systems end. But you will anyway at some point, and making that hire earlier can pay for itself.
Having your own equipment is a CAPEX and investors don't like to see those on a balance sheet at all due to various accounting reasons. Mostly its seen as a burden. Cloud is an OPEX and investors seem to prefer renting to owning.
Personally I don't understand why spending 3x more is more attractive to investors, but often the technical reason being right is superseded by the business logic.
Yes, queue the comments about "Total cost of ownership", past the point where you cant afford an OPs person(s) (which you will eventually need for AWS anyways) AWS is a money-sucking black hole.