I don't mean installing the software, I mean hardening the box so it's not a sitting duck. It takes time. Sure, there are fire-and-forget ways to setup mail software like Roundcube, but I am referring to VPSes themselves. There are innumerable things that can go wrong. I will outline a two of the most (usually overlooked) ones:

Origin Shield: You don't want the raw IP of a box anywhere near the public Internet. Attackers can scan for IPs if they want an IP, but they should never have to simply ping a server and its IP is in plain sight.

Auto-update: The box should respond to the current threat landscape. There was a massive uptick in sysadmins logging into old and forgotten boxes when Heartbleed came out. You want to be able to auto-patch and very rarely (if ever) SSH into a box.

Those are just two of the things I've learned the hard way recently, and yeah; they're probably obvious low hanging fruit stuff. But the list goes on...