With the HA design you tend to need running on AWS for serious operations (AZs out the wazoo with every other feature tacking on a few dollars at a time), you might as well use multiple cloud hosting providers and load balance across them. Obviously this isn't typical practice, but Amazon's lock-in is pretty glaring when looking at one of the best potential aspects of cloud being that of PORTABLE workloads.
for a small webpage/server, perhaps linode or digital ocean. I find heroku nice for free hosting, but they are a bit pricy for paid services. I do love how easy it is to push code to heroku (especially with ruby on rails). if you never used amazon and only want to use it for a year, their micro instance is free for one year.
for large scaling systems and a suite of services, I would suggest AWS. they have so many services it confuses the heck out of me sometimes. if you want low cost high performance, especially for development digital ocean seems better for this.
i never used Azure, cant comment on it. Also there was a company that basically rented physical servers at a cheap price, from what I remember they had the best performance per cost that I have seen. Also another obvious consideration is software/applications - some tie in better than others. for instance, I use ArcGIS Server which is VERY easy to deploy on AWS
they do save a lot of money if you do not need to hire certain types of IT fokes, and aws is extremely cost effective for satisfying the need when you have low traffic and then sudden large bursts of it that must be handled.