The real difference is not in price; it's that in shared hosting you don't get the option and associated responsibility of managing your software stack; in VPS you do.
They also give you some ability to "scale up" by simply asking for a bigger instance, and they handle moving your server over.
The only real downside is the default password. Unlike AWS, you are given a root password, which is pathetically easy to crack, so the first thing you need to do is change that.
Hell, my dad uses godaddy of all things, but at least he has his tiny wordpress site up to help his non-tech business.
Also, Dreamhost at least gives you shell access, which is a big plus. And, contrary to a comment above, you can build your own environment. I have an up to date python 2.7 virtualenv from easy_install, and same for ruby.
However the cheapest linode vps is $19.99/mo.
I would LOVE to have a vps to play around on, I would have so much use for it and yet I'm a student living in London and all my money goes towards the cost of living. It makes much more sense for me to go with the cheaper shared hosting as I just can't afford anything more.
As an aside, I found http://virpus.com/ the other day and they are selling vps's starting at $3/mo. Does anyone have any experience at all with them?