(Part of me is wondering if many of the more controversial discussions on this site are between people who have sysadmin experience (and considered it valuable) and people who don't.)
I've done a fair bit of sysadmin work over the years, mostly in self-defense, because I want fewer crises when a critical development server eats itself.
Lots of people can figure out how to install Ubuntu, or rent a Linode. But if they don't master upgrades, patches and backups, they'll eventually end up paying a real sysadmin a lot of money at the worst possible moment.
point 3 isn't that necessary if all you're doing is git over ssh. All the people that have a checked out copy have a backup of your repo. Also, if you're only doing git over ssh then it's not hard to make the box relatively secure. ssh will be hte only open port, and it won't be on the standard port.
1) You need to keep current on your security patches.
2) You need to upgrade your OS when it reaches end-of-life.
3) You need to make backups, and more importantly, verify that those backups will actually restore. (For git, this is most critical if you have a large team and dozens of repos.)
You can ignore a server for years, but eventually you'll get compromised or loose a hard drive. (If you use RAID, you'll eventually lose or an entire RAID array. Not fun at all, nor cheap.)