It is a bad idea to host your own SMTP, but only because you need to keep on top of maintaining it. Most of that effort is the same if you have one user or a million, so it is more efficient to outsource that maintenance. However if you are willing to pay the price in time there is no reason you shouldn't be able to maintain your own server.

I use fastmail. They have humans that respond quickly when I have a problem (twice in 10 years)