One of the most amazing feats Amazon has pulled off is to convince people that AWS is cheap. They're cheap in the way that Apple are: Only if you need a feature-set (or name recognition..) that excludes the vast majority of the competitors from consideration. If/when you truly need that, then they're the right choice. There are plenty of valid reason to pick AWS.
But they're very rarely the cheap choice.
Yes, but that's a false comparison. It's cheaper to rent dedicated servers at any of several dozens large hosting providers than it is to use EC2 or S3, for example. For most people it's cheaper to rent racks and lease servers too (but depending on your location, renting dedicated servers somewhere else might be cheaper - e.g. racks in London are too expensive to compete with renting servers from Hetzner most of the time, for example).
It's extremely rare, and generally requires very specific needs, that AWS comes out cheap enough to even be witting batting range of dedicates solutions when I price out systems.
When clients pick AWS, it's so far never been because it's been cheap, but because they sometimes value the reputation or value the feature set, and that's a perfectly fine reason to pick AWS.
The point isn't that people shouldn't use AWS, but that if people thing AWS is cheap, in my experience it means they usually haven't costed out alternatives.
It's an amazing testament to the brand building and marketing department of Amazon more than anything else.
E.g. my object storage costs are 1/3 of AWS. My bandwidth costs are 1/50th or so of AWS prices.
There are valid reasons to use AWS depending on what exactly you do, but it's extremely rare for price to be one of them.