You're glossing over the complexity of configuring email solutions. Though I'm not entirely certain if you're only advocating web hosting on a VPS with mail through something like Google Apps for Domains.

Things like Exim and Postfix are non-trivial to setup correctly and very easy to screw up the configs, at least in my experience; though most of the time that I've tried to configure them I was trying to get them to do non-standard things (like redirect all outgoing emails to non-whitelisted -- through a pattern/regex -- email addresses to a specific mailbox).