Beware of super cheap VPS providers. A couple years ago I got burned by a provider called VBOX that was highly recommended on lowendbox.com. At first the service was ok, but a few months later my server and VBOX business were completely unresponsive.

Sure you'll save some money, but you might also waste hours of your time setting up a server that could one day disappear into thin air.

So long as you have deployment automated, and frequent backups, that shouldn't be a huge issue. One of the many advantages of not having to manually deploy a new box is being able to rapidly get off a bad provider.