Running your own mail server does require investing some time in learning the software, the protocols and all the ways spam filters will sabotage your attempts to send legitimate, non-bulk e-mail, as well as a little time for maintenance, but it's really not as bad as people say, at least in my experience. It's not something you "just" do, but it is doable.
More generally:
- I made sure my server's IP addresses weren't on any blacklists.
- I host my mail server with a reputable hosting provider (RamNode). Stay clear of companies like OVH that have reputations for poor abuse handling.
- I have configured SPF, DKIM, and DMARC.
- I registered my server with https://www.dnswl.org/, a whitelist for legitimate mail servers.
- I don't send spam or other bulk e-mail, only personal correspondence.
I also tried sending e-mails to a couple of people I know who use Gmail. I asked them to mark my mails as 'not spam' if they went into their junk folders. I don't know how much this helps "globally" in the Gmail system, but at least it'll teach Gmail not to mark my mails as spam for those people.
I had used my current personal domain for e-mail for a couple of years but I paid a company to host my e-mail for me. It might have helped that my domain had already built up a good reputation in big mail companies' spam filters, but who knows... It definitely helps, however, that my domain wasn't brand new. Newly registered domains are often penalized by spam filters. A poor choice of TLD can also hurt delivery: I know that many mail admins penalize or out-right block many of the new gTLDs like .top and .download.
I am not discounting those who complain about deliverability, but so far I haven't had much trouble with my low-volume, personal mail server.
I don't think we should dissuade people from doing it, especially if the fact that more people doing it means that it'll be easier next time because it'll be slightly more common. Many of us are in tech and the field is a small subset of the population. Even if it's a small amount of servers setup by us, that could make a noticeable on those working at bigcorps who write the hostile receivers.
Setting up is easy, maintaining and responding to all the blocks you may get is ongoing and simply cumbersome. That is what people are referring to when they say its tricky.
You may wake up one morning, and find your host is blocked through association (if you are lucky - some hosts will simply silently swallow your mail - i.e. it never arrives and you dont know about it). You can apply for removal in some cases (and wait for it to update before you resend), and in other cases find your hosts entire subnet is blocked, resulting in you having to set up another server on a different subnet in order to relay your email.
There are ways around a lot of this, but hosting on AWS/DO/Etc not having your own assigned subnet, etc, will most definitely result in the above when you least want it to happen.
Of course SPF and DKIM were supposed to alleviate the need for these IP based blocks, but the reality is that they haven't changed a thing when it comes to sending email to large hosts.
Thats why we say its tricky.
The only real issues I had were with Gmail for a few months, until they got used to the IP address sending legit email. You have to instruct people to look in their spam folder (of which Gmail hides the existence, an evil thing in itself) until people marked like 10 emails as being non-spam over the course of a few months. Hardly had any trouble otherwise, and I don't even use DKIM nor a reverse DNS for all domains (only SPF because that was easy to do, and with 1 IP address I can do only 1 reverse DNS, and only v4 at that).
It helps that I've got a static address, and it might also help that XS4ALL (ISP) has a good abuse team that probably keeps the netblock quite clean. I don't know to what degree this affected my experience.
You're not just setting up mail, you're dealing with an email cartel that does not have any desire for small players to exist. Note that I said "small players", not "spammers". Enough spam makes it to its destination that there are certainly palms being greased.
No need to get snarky. I do run a personal mail server myself so I'm reasonably sure I know what reality is like.
I also have a personal mail server that works fine, but I wouldn't recommend most people to do it, because getting spamholed is quite a serious liability.
You could attempt to limit the outgoing emails per account per hour, however if that's set too low then you end up with other users who can't send out emails to their "mailing lists" consisting of hundreds of contacts, instead of using a real mailing list to manage it.
The effort just isn't worth it.
Is this an urban legend that just keeps getting repeated?
Hashcash, Bitcoin's PoW game, was intended for use as an email attachment to create a real cost to spam without any micropayments.
"If you want to mail me, prove that you burned five seconds of CPU time."
I don't know if anything like SPEWS is still around but they used to do large blocks. I still don't know if it was genuine attempt to reduce spam, or performance art, or god-tier trolling.
https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_...
https://web-beta.archive.org/web/20030609200813/http://www.s...
That’s why
What is the motivation for the "guilty by association"? Because you can not verify who sent it.
With SPF and DKIM you can - that is the whole point, you know beyond any doubt that the machine supposed to send that mail actually did send it.
Now, if the person behind that machine is a spammer (or has been hijacked by a spammer) and sends out a lot of spam (and gets reported as such) then just nuke that sender. It is dead easy to do regardless of what IP it was sent from - each mail have identification credentials built in. It could not get any easier to do. The sender did all of the work for you by going through the extra effort of identifying itself.
Yes, it'd be nice if they didn't do that, but small sites are and will be acceptable collateral damage to them.
I don't know who you're even trying to convince. You're talking in hypotheticals and being extremely idealistic. How the email ecosystem works and how it actually works are two very different things. You cannot say "the reason must be something else!" as if - just because you can't make sense of the situation - it is not so.
Not whatever IP was used.
I'm not being idealistic when I say that I should be able to send an email over internet and have it delivered without someone banning me for reasons beyond my control.
That could be said about lots of things, but this is kind of remarkable in how much effort is being made to make something as bad as possible.
One that has been given in the past is "Stop paying money to companies that host spammer websites and spammer email servers, or we'll block your email too"
(I don't particularly agree with it, but it was common to see that argument on news.admin.net-abuse.email
Sometimes they claim to be real people with real e-mail addresses that they don’t actually own, and maybe that real person is you — in which case it is your reputation that gets permanently dinged.
Sometimes they claim to be fake people with fake e-mail addresses that don’t actually exist, in which case all you really know is the IP address they used to send the message. When they do this, they frequently arrange to get a large block of IP addresses they can use, and then cycle through them very quickly — maybe no more than five minutes per IP address. Anyone else who happens to be in that same range now gets their reputation dinged by association.
There’s a lot of tricks spammers use to try to hide their activity. And lots of ways to try to detect and reject that on the other end. It’s a never-ending arms race, and on the Internet it is easy to hide and hard to take out miscreants without causing excessive collateral damage.
And many big providers don’t care if they cause collateral damage.
Now, if SPF and DKIM has not been configured then fine. That is something else, resort to whatever technique you have to.
A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.
Also, it isn't in said idiot's interest to deliver your email properly. They would rather fart around with machine learning and blame whatever happens on incomplete training data...
You can do "everything right" rolling your own email, but you are still relying on the receiving parties to do their job and implement sane spam prevention policies. This is far from what actually happens.
I lead development on a project that sends a lot of (legitimate, user-requested, transactional) email to many mailboxes belonging to a certain government department. I used Amazon SES as they have a pretty good track record. The receiving mail servers were configured to use a few overzealous blacklists that were causing my email to be silently swallowed (and reported as delivered). This was based on an IP black-level ban. Simply Googling the names of these blacklists showed report after report after report of them being completely useless, with many seasoned sysadmins saying that nobody worth their salt should pay any attention to them.
This was extremely hard to troubleshoot as this department was not a direct client and was of limited helpfulness. It became apparent that using SES's newly-launched dedicated IP functionality wouldn't have helped. I made full use of my AWS Business Support, the multiple people I talked to made it clear that SES put a great deal of effort into ensuring their IP ranges are "clean" - but there are some spam lists that do not play ball. The most important takeaway was that I was doing everything right. I was using a reputable mail provider who have very stringent spam prevention controls. I utilised SPF and DKIM. The content of my mail shouldn't have (and didn't) trigger a content-based spam detector. It was because the people on the other side weren't doing their job to an acceptable standard. When I was eventually able to open a channel of contact with the receiving department, they confirmed it was an IP range-based ban.
The fact that I was doing everything in my power to do the right thing had no effect on the situation. I still wasn't getting mail to customers. Saying "darm, these sysadmins really aren't doing their job properly!" had no effect on the situation at all.
We still live in the age of almost-but-not-quite-plug-and-play email solutions like Exchange that can be configured by people that probably shouldn't be configuring mail servers. For all I know, there was a list of blacklists on some Hosted Outlook control panel and the person in charge ticked all of them because more is better. As long as this is the reality I live in, I'm going to do all I can to play ball and succumb to the black magic. I can imagine my experience would've been a lot worse if I had been operating a mail server out of a dirtier IP range (generic cloud hosting provider, retail ISP customer IP range.etc). If I recall correctly, AWS severely limits sending mail from EC2 instances because they don't want their IP addresses / IP ranges to end up on mail blacklists.
On the other hand, if you absolutely depend on the ability to send emails such that your recipients reliably get them, hosting your own email server is extremely tricky. One could carefully go through the checklist of SPF, DKIM, ip blacklists, etc and emails will still be rejected by MS Hotmail, GMail, Yahoo, etc. Those giants are "black boxes" when it comes to their heuristics for rejecting incoming mail as spam. E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". Trying to debug your "sender reputation" is not easy.
In the 1990s, I hosted my own home email server over ISDN lines. These days, I have a million other things I'd rather do than babysit a personal email server with software updates, SpamAssassin lists, etc. I get the whole decentralized ethos but it's just not worth the effort for email servers.