And where does that new server come from? How quickly can you get one set up at your datacenter? Automated software deployment is easy by comparison. Hardware failures are rare until they happen. At the end of the day, you get to decide if the value of business continuity is worth it of not. Maybe in your case it's not.

I'm not suggesting that VPS is the answer for everyone though. Certainly an app that has significant infrastructure almost always benefits from rolling their own hardware, and hiring systems administrators to manage that infrastructure.

Since you're talking about a single machine, from my perspective, a VPS is cheaper and more reliable over the long term.