Edit: NVM. I see Fastmail when I reread the comment.
Is there a way to only host the receive portion?
I'm happy to pay someone to handle all the idiocy around sending email and getting it through Google and Microsoft, but I'd really like to hold my emails myself.
Sure. Set your MX to your own SMTP server but pay a mail delivery service to send your emails & use their SMTP servers as your outgoing server. You'll have to setup SPF & DKIM appropriately of course.
It's not trivial to do this, but it should be possible.
Is there a paid mail delivery service that doesn't get marked as spam instantaneously?
Or is there a way to do this through something like Fastmail?
All of that seems easier than setting up a server to keep your e-mails.
Sure. Set up your email server and have DNS of your domain point the MX record at it so you receive all email directly without any third party involvement. Then, set up outgoing email to forward to some third party which handles delivery for you. IIRC there are some free options even, if your outgoing volume is low.
That said, my email server handles incoming and outgoing email for a handful of domains for self and family, no problem.
Google: anonymous inscrutable guillotine
Fastmail: payments fail and I do not notice for too long
It's certainly my biggest worry about using them.
(I didn't want to actually host my own mail stack, so I just have a custom domain set up with fastmail and point the MX to them. Their UI is great and a breath of fresh air compared to gmail. I guess they could in theory decide to lock me out randomly too, though I trust them to have actual customer support and can just point the MX somewhere else in the worst case)