The problem is that staff time costs an order of magnitude more and most of these things can go wrong. For example, you don’t just want to buy random lots of whatever disks are cheapest because then you have to track recalls and firmware updates for everything.

If you have two tiers available, someone will use the less reliable one because that’s how they fit their budget but then because it’s “in production” they’ll expect the same level of service.

All of these are manageable but what you’re really hearing is that the technical issues are really the tip of the social iceberg most organizations have. One of the reasons people pick AWS isn’t just that it’s usually cheaper than the full cost of rolling your own but that lots of these things don’t affect you: you never fail to provision an EBS volume because the VPs of finance and IT are still arguing about procuring a new rack of disks, people can’t request endless customizations because the options are “take it or leave it”, etc.