Step 4: wonder why the hell people aren't replying to your mails. Step 5: realize you've been blackholed by gmail and hotmail with no bounces and no explanation. Step 6: frustration. Step 7: tell people to add your e-mail address to their address book (wtf) and watch as things magically work for a while. Step 8: repeat.

My mail server routes via Amazon's SES, which provides an SMTP smarthost my instance of postfix can relay mail to. You authenticate each sending domain using DNS, and it even supports DKIM signing.

Until I did this, my deliverability, especially to GMail, was awful.

Assuming you're running your own mailserver because you like the idea of a decentralized web; isn't it kind of odd to then rely on Amazon?

Maybe, but in my case it would not be a valid assumption.

Have you seen any issues with the envelope from not matching the the email address?

ezmlm, for example, will not let you subscribe/unsubscribe

I've not run into any problems myself - even if I did, I think I'd rather be able to reliably email people with GMail accounts!

That's a smart way to work around a frustrating problem. It's a bummer there are so many loopholes to jump through.

I don't know why so many people on HN tell this story. Are the US IPs all total garbage?

I set up my mailserver, sent a testmail to Google, no spam or other stuff. I don't know if SPF or DKIM was even configured. (It now is of course)

Do you have a fixed or dynamic IP? When I tried sending email from my home network (which had a dynamic, if long-lasting IP), I discovered my ISP's addresses where on the PBL[1] and therefore getting rejected by many servers. This was in Europe, not the US.

[1] https://www.spamhaus.org/pbl/

SE Asia here and many IP blocks (/19) of my ISP are blacklisted, even blocks those are being used in datacenter in that bigger block.

/29

And of course, not hosted at home.

I'm in the US and I don't get this story either. I've had my mail server running on 3 different IPs, in my home, and have had no issues at all.

Heh. I think your steps 4-8 are all just “step 3 learning pains”. Configuring DNS for a “real email system” is often not trivial to figure out how to do. The steps are usually pretty “easy” (though getting a PTR record sometimes isn’t), but are frequently overlooked by beginners.

I wish that were the case.

I've been administering mail servers going back over 20 years and I wouldn't recommend it to anyone. The last 4 years or so, things have gotten a lot worse. Google and Microsoft are actively and deliberately making it harder for people to run their own mail servers.

Sure, setting up DKIM and SPF and whatnot isn't hard. But blackholing legitimate mail without any reason, without any form of notification or any possible appeal, well that's just appalling. And sure, Google & MS have tools to debug a subset of mail issues, but those don't even work unless your MTA is sending boatloads of mail to their servers.

This is exactly the problem you run into when sending mails to Gmail: For them there are only bulk senders and gmail address owners. In their whole documentation / faq they don’t even mention small businesses or regular people owning a mailserver to just get contact other human beings.

People at Google might be the smartest under the sun, but they don’t understand email: It’s primarily a way for people to communicate with each other. In Google’s eyes it’s only a way to deliver advertising to end users.

Edited for typo

> People at Google might be the smartest under the sun, but they don’t understand email

Oh no, they understand email very well. They're just arseholes who want to force "regular joe" and small business to use their services. This isn't some kind of accidental incompetence. There's no "whoops, sorry your mail isn't getting delivered, our bad!"-type situation going on. This is a deliberate strategy of making it as hard as possible for people to run their own mail servers.

Of course they hide behind the excuse that you might be a spammer. But given Google's massive resources, there's no way in hell they can't tell ham from spam.

Yeah. Not to mention a lot of SMTP pain is when a major provider decides to blackhole you, or you are spam blacklisted, neither of which is under your control. Even a large corporation that self-hosts mail may have to constantly call other companies to get on their good side so they can send mail again. (The bigger you are, the more often this happens, but it also happens to small timers)

I think Step 7 is a good idea. Initially, I asked some contacts to send me an email from Gmail and Protonmail, to which I replied. From then on, my emails work fine with them (negative spam score).

If you don't want to do everything by hand, check out https://github.com/sovereign/sovereign/

I'd rather not run heavy opaque Ansible magic that also adds IRC bouncer and checks kitchen sink to configure e-mail server.

All those features are in separate small files and have tags. You can enable and disable them one by one.

Yes, and for junk I chose rspamd for Bayes classification, very effective so far.