VPSs are great, but I've never got the point of things like engineyard where you're paying masses to get people to do sysadmin.
There's so many reasons a site can be down, you can't just move all responsibility to another company - at least I don't think that's a good idea at all. By all means move hardware to them, net connection to them. You don't want to worry about power outages, etc
Keeping a competent sysadmin around who will reliably respond 24/7 is much more expensive than managed hosting, in general.
You think you can do it, as a founder, especially when you've got some sysadmin skills, but you simply cannot do as good a job as a team of operations people who are stand-by 24/7. And you simply cannot afford such a team for yourself when you're starting out.
And that's where you make the mistake. Look at twitter. Look at facebook. Look at anything else. They're all broken from time to time, all offline every so often. People are pretty forgiving especially at the start.
"Would you pay $100 an hour for an untrained accountant? Because if your consulting rate is $100 an hour and you do your own accounting, that's exactly what's happening." https://gist.github.com/0a2655aed6a26fa15a02
So I guess the relevant questions are:
1. How much is your time worth?
2. How much are you willing to pay for an untrained sysadmin?