Is Heroku sufficient? It doesn't have the email aspect but there are plenty of standalone options there. It abstracts away a lot of the headache from custom VPSen, as long as the sort of thing you want to host is supported. AWS OpsWorks could be an alternative.