I'm a big proponent of NearlyFreeSpeech (http://www.nearlyfreespeech.net/). Everything is pay-as-you go, literally. 1GB data transfer = $1. Everything is super affordable, but let me state this, and this is the theme of what I'm going to tell you: it can become expensive if you're running a very trafficked site with lots of user activity.

That said, you should look at hosting with price as the very last thought. Find out what you need, compare that against availability. Find out how much you need, compare that against reliability. Find out how you can support it, compare that against your host's support community, staff and users alike. Using things like this you pick out your host, and if the price is okay, great. If not, move to the next and deduce in the same fashion.

EC2 I think is a good way to go to keep prices low to start off, and if you want to something else later on you'll have the finances saved to do so. Just know the more you use, the higher costs will be, but at least you're not paying $40/mo for clustered/redundant/balanced hosts and databases when all you're doing is writing a blog early on in the game.