I have a domain, serviced by a registrar. When I change email providers, I go to the registrar's site and point my email address at my new provider. That might take a little time (way less than a day, by observation) to propagate. I then go to my new provider and set things up to "send mail as me@mydomain." If you run a local client like Thunderbird (rather than run exclusively through your provider's web interface), then you'll need to do similar there (like point it at your provider's smtp and imap/pop3 servers).

Problems arise when you run your own server, because of fairly detailed authentication settings. If you pay a provider to run their servers, they take care of that.

I do that. It's actually not too bad, I think the occasional missing email is overblown. At least, I don't typically have problems.

Of course, it's important to keep backups of everything (configs, logs, mail, etc), and test. Just slowly port everything over, as you gain more confidence that your not missing anything. Most email providers also will let you put a rule in place that lets you bounce mail to both addresses.

Hosting your email with an established provider is easy, set up SPF records and the existing IP reputation of your provider will get you past spam filters quite easily.

Where you run into problems is when you do what I just did, I was using Google Apps, then Outlook.com with a custom domain and decided I was tired of giving other people access to my data - since I have a decent business-class cable connection at home including a static IP I bought a couple GroupWise licenses for myself, my wife and my daughter and moved our email in-home where all our data sits comfortably on my FreeNAS storage array. Since I rarely send email, and my home IP is 'unknown' to any IP reputation services at the moment getting past spam filters will require some time. It's a non-issue for me, it's a personal email account and I can tell my friends and family to check their spam folders the first time I contact them to whitelist my address, but it's not something everyone wants to deal with.

I had my own domain-based email address rerouted to yahoo email. Started to get a LOT of spam that would not get filtered out at all, not even close from the the yahoo-based email filtering. Gave up the forward. Maybe should try gmail?

Hosting was pretty easy with https://github.com/tomav/docker-mailserver

Tho FWIW I use Postmark to do my email delivery because then at least I know my email will end up in the inbox of the recipient vs their spam box which my hosted server was having problems with.

Running your own mail server is a pain. Paying for google apps to handle your email is very easy. Other providers can do the same thing for you.