The problem is... it sucks. I'm doing this and - honestly - it's not very pleasant experience.
The dilemma is - either sell your soul to Google (Apple, Microsoft, whoever else) and surrender to their will for a glimpse of their proprietary innovations, or have to deal with basic (and somewhat ancient) technologies without much of support. Unless you hire a sysadmin to maintain that for you.
The thing is, popular services add some value, like smart email processing, reasonably-well maintained spam filtering, tagging emails (IMAP doesn't have this!) etc etc. It's all theoretically doable in DIY setups, of course, but entry barriers (time, knowledge) are huge, and in practice one'll likely to end up with only basic functionality that'd they'll end up eventually abandoning due to maintenance being a PITA.
</rant>
This, of course, requires that everyone use the hosted email client on that server, but it's worth remembering.
Personally, I (and everyone at rsync.net) just use (al)pine, so the hosted email client is very simple (and fast and efficient).
Or at least they log the ipv6 route within their own datacenter, or between datacenters.
So that is up for argument if it is traveling via their own intranet or "the internet".
(at least google uses AES TLS, yahoo still uses RC4)
(and if you are not in the usa, just assume it is realtime)
This is why everyone should probably have their own private email server locked down in their home. Postfix can be made pretty secure.