This is untrue. If you are the only person using your email server, your volume will be so low that the big providers (Gmail, Outlook, etc.) won't track your reputation. So, ironically, being a low-volume sender means your email will be constantly classified as spam.
I speak from experience: https://www.attejuvonen.fi/dont-send-email-from-your-own-ser...
My email server is used by two people. Reputation is tracked by all the big providers, as evidenced by a) my email not being classified as spam, and b) them showing reputation of my domains in their various reputation dashboards.
"Those who say it cannot be done should not interrupt those that are doing it."
When you make a claim that supposedly applies to all people, a single counterpoint is sufficient to disprove the claim. It's as if you had said "all rabbits are black", then I showed you a white rabbit to counter that not all rabbits are black, and you come back with "look, I have a black rabbit here". How does that make sense to you?
> them showing reputation of my domains in their various reputation dashboards.
I never got access to their dashboards because my email volume was so low. If you somehow did, good for you.
> "Those who say it cannot be done should not interrupt those that are doing it."
I'm not "interrupting you from doing it". I'm interrupting you from giving bad advice to OTHER people.
Well said!
Your claim is "it is not possible to self-host your own mail on a low-volume server and not get consistently marked as spam by GMail / other large operators". The existence of a single person successfully doing exactly that (and there are numerous such people in this very thread) is sufficient to disprove your claim.
Perhaps that was their claim - but I've generally read advice as: "There's no predictable way to guarantee that any given person can today take over hosting their own mail with predictable and good delivery to Gmail and o365."
So just that a, b and c have, so far, good delivery from their setup is not a guarantee that person x can just "set things up correctly" and somewhat straightforwardly get good delivery.
Last I did it, I had to go via undocumented api/pages for both o365 and Gmail in order to improve delivery - and mail that gmail/o365 smtp servers swore they accepted without problems - still sometimes ended up as spam, or simply vanished after delivery.
This was all individual low-volume. Never found any reason for it.
That said, I'll probably go back to hosting my own mail, and just live with certain parties being bad net citizens, eating the occasional mail without error or bounce. It's not like I really expect them to do better. Although especially in the case of Gmail, it's a little like Disney eating up public domain stories and spitting out copyrighted and trademarked content. Google did a lot to force people away from proper quoting (by hiding the fact of how Gmail quoted things in the "friendly" ui) and they pretty much killed Google groups - after marginalizing alternatives. But those ships have sailed.
I do think that most of the effort/risk is at the beginning. Making sure you're on a reputable provider, checking the history of your IP, setting your mail server & the security features up correctly, monitoring deliverability etc.
After everything is working well, if you got that part right, the ongoing effort should basically just be keeping software up-to-date. You could always get unlucky and e.g. someone starts sending spam on a nearby IP and you have to waste some time dealing with that, but hopefully if you picked your provider well that won't happen. It's yet to happen to me, but my provider only offers dedicated servers, which are probably not so popular with spammers.
Oh, that tiny little detail where it has become impossible for almost anyone to actually send email from home, either because residential IPs are all marked as spam, or because your provider is giving you an incomplete internet connection that blocks outgoing SMTP.
> You could always get unlucky and e.g. someone starts sending spam on a nearby IP
Can someone tell me why providers ever thought it was a good idea to block entire IP ranges?
Well, by "provider" I was referring to the hosting provider. I wouldn't typically recommend hosting from home [0]. Apart from the issues you identified, over the course of 20 years I've moved house multiple times so would have had email downtime and likely had to change IP address, likely leading to reputation starting again from zero (at best) or picking up an IP with bad history (much worse).
> Can someone tell me why providers ever thought it was a good idea to block entire IP ranges?
If you repeatedly receive spam from multiple IPs within a given ASN, and the abuse contact is non-functional or doesn't actually resolve the problem, it is fairly reasonable to consider that the ASN is friendly to spammers. Blocking by prefix rather than entire ASN is a bit less of a sledgehammer, but follows the same logic.
[0] If you own your house, have no plans to move any time soon, and can get a connection from a provider who is willing to give you a stable static IP that isn't categorised as 'residential/dynamic' by the major lists, hosting from home should be fine. But that's quite a few 'if's.
"will" is a strong word. I've read that very low volume sending server can sometimes have issues, but never experienced it. My outgoing volume is about as low as it gets since it's just me and some family that don't use it much, but don't experience any problems.
More likely, they're saying that 99% of people don't know how to self-host, and for 99% of the rest it's not worth the trouble. Also, if you have to ask, then you shouldn't self-host it.
For a second, I thought I was on Stackoverflow. If you aren't starting by asking questions about the possibilities or limitations of a system you're about to work in, then you aren't starting properly.
We need to give newcomers a break and answer their questions well, and discuss to promote understanding, instead of swatting at them with our canes. The only way knowledge passes to the next generation of thinkers and tinkers is if we fuel that curiousity.
There's plenty of other tech they can screw with.
I've wanted to do this for a long time, and am using this very HN post as a source of information. I don't expect anyone to set it up for me, but I am hoping to identify the "gotchas" that are likely not discoverable by reading man pages.
If you start self-hosting now, you should be prepared to lose quite a few emails randomly for the first X months while everyone else tries to figure out whether you're legit or not. Though I would encourage anyone who can to try to self-host at least some part of their email infrastructure, even if just for the learning experience, I would also recommend that they avoid using self-hosted email for anything business-critical until they're sure they've got the hang of it.
Use mail-tester.com or similar tools to ensure everything is configured correctly.
And then just start sending. As long as your volume grows slowly over the first few months, you’ll get basically no rejects.
And then prepared to lose quite a few emails consistently for the next 10 years when some decide you're not legit.
Source: I self-host.
Also, little-known fact: if you register a UK company (probably more practical if you already have one, but the effort is not actually that big), you can register .uk domains directly with Nominet, the UK registry, by setting yourself up as a self-managed registrar. It doesn't cost anything (beyond the cost of the domain name) and is very easy. I'd love to know if there are any other registries that allow something similar.
Certainly way more than 99% of the general population wouldn't know how to self host, but within a techie population like HN, easily ~50% can be capable of doing it if they wanted to. Whether it's worth the effort is a personal decision, but there's a lot of value in owning your own email so I recommend it to anyone who's curious about it and willing to do it.
> Also, if you have to ask, then you shouldn't self-host it.
We should be encouraging curiousity (a HN value) not stomping on it.
If anyone asks, I say go for it. Worst case you'll learn new things, best case now you own your email.
Otherwise, how would anyone learn anything?
I think it's also fair to say that personal mail for a small domain is much easier than even a small amount of transactional email and don't even try sending newsletters beyond your friend group.
I have run mail off three different IPs over the ~20 years I've been hosting, switching IP address didn't affect me all that much.
Another thing to note is that receiving mail is really easy. Sending it is hard, filtering out the spam (and only the spam) from your inbound email is harder.
Of course, this is not 100% reliable, as it's not too difficult for spammers to adapt and improve their scripts. Of course, vast majority of spammers are either not sophisticated enough, or do not care enough to do so, so if you don't mind your incoming mail to be slightly delayed, it's kind of a low-hanging fruit, as it cuts off a huge amount of low-effort spammers.
blackhole IN A 240.0.0.1
@ MX 10 blackhole.example.com
@ MX 20 mail.example.comDoing it this way doesn't even delay mail much most of the time; many legitimate MTAs connect immediately to the priority-20 MX after failing to connect to the priority-10 one.
So a easy way to get started is to receive everything directly and use a commercial (often with low-volume free tier service) relay for outbound until you get comfortable enough to remove the training wheels. (Or never remove them, that's a legit choice as well.)
> filtering out the spam (and only the spam) from your inbound email is harder.
I don't find that at all. Filtering spam is the easiest part. All I do is if SPF doesn't match, goes to spam folder. Beyond that, apply a bayesian filter.
I get no false positives and the spam that gets through to my inbox can be counted on one hand per quarter. Basically none.
That's yet another benefit of self hosting, since my bayesian filter is trained on my personal email specifically, it tends to become very good. Unlike generic gmail filters for example, where there'll always be some mail that ends up in spam no matter how many hundreds of times you mark it not-spam.
I have Bayesian filtering behind DMARC checks and a few other DNS and RBL checks. Still not perfect -- for all folk talk of the difficulties getting mail accepted by the big providers, a surprising number of companies still don't have matching forward and reverse DNS records :P.
One principle I follow is that I don't drop anything. Every message will either be rejected or delivered, with a narrow band of SpamAssassin scores that delivers to a spam folder.
At the very least, getting your server marked as spam/blacklisted is not inevitable. Just make sure you aren't an open relay and that you've got properly configured SPF and DKIM records in your DNS. Once that's set up you can pretty much forget about it. I haven't had to touch any of my configs in years.
Initial setup takes maybe a day or two if you know your way around Linux or one of the BSDs.
The only nag is that Microsoft is EXTREMELY strict for their hosted email. It's the only provider that consistently denies recieving mail from my server when the IP range it's on becomes greylisted in UCE-PROTECT -- which happens every so often...
Easily solved with getting the MX and backup MX IP's whitelisted there, but I haven't bothered cashing out for that yet...
This is a common complaint with Linode specifically, but probably fairly common with low-cost virtual server providers in general. It's worth looking into the history of your IP before you start using it to host mail, and if it's feasible, shelling out for a dedicated server (ideally from a provider that doesn't also offer virtual servers, or has enough network separation between them) makes it much less likely that your neighbour is a spammer. Mine's never been on UCE-PROTECT.
2007..2014 were probably the worst. Gmail was chainging often, Microsoft was blocking everyone.
I think self-hosting is easier now than 10 years ago.
This is because the business department will try to force admins to ignore requests to remove spammers as the spammers generally pay decent money.
By blocking everyone, it forces the ISP to get off their ass and kill the spammer irrespective of how much they are willing to pay.
https://blog.sucuri.net/2021/02/uceprotect-when-rbls-go-bad....
ISPs are now so sensitive to not allowing spammers that the RBLs can't make money from it anymore.
This idea that self-hosted email is impossible is wildly overblown.
I've helped others set up self hosting much more recently, and haven't had any reputation problems beyond the early period where the IP has no history. (It is important to find an IP that doesn't have recent bad reputation, but that is fairly easy to do. Unless your host is in the business of hosting spammers most IPs will be clean.)
The reality is honestly just that self-hosting mail is not as hard as all the people who don't do it say it is.
Your ip and those of others who have sent mail to gmail have been recorded.
Your reputation score is high.
Try a new ip and see how hard it is.
The other option is to relay through Send in Blue, or Sendgrid or something like that.
No, it's not "wild".
Its just that we're in 2022, not 1997.
Long gone are the days of "fire up Sendmail and you're good to go".
To those thinking of self-hosting, I would say they should start by understanding modern anti-spam.
Understanding modern anti-spam will not only help them with their inbound email, but will also help them understand how to ensure deliverability of their outbound email too.
You don't even need to set everything right. Up until very recently (months), I was sending emails from a few of my servers, and I had NOTHING set right. As in, I was sending from IP addresses that were never mentioned on my DNS, no PTR no SPF no DKIM no nothing. Just good old "here's an email from this address, trust me I actually own that address and it's legit".
And it worked just fine.
Surely just a reputation thing, as I had been doing this for over a decade, and all emails were very important (password recovery, order details, etc), no newsletter or anything.
I recently replaced all that with zoho because I wanted something a bit more secure and didn't want to configure it myself.
Just because you're lucky doesn't mean other people are "unexperienced". I don't know what experience has to do with it tbh. Maybe if you have experience working for Microsoft and you can contact the right people there...
In my experience the difference in outcomes that people have with self-hosting mail are largely down to, from most important to least, (1) the nature of the mail they are sending, (2) their choice of hosting provider, and (3) the correctness of their mail server setup. It sounds like you've got (3) sorted, so maybe (1) or (2) are relevant here.
…except when Hotmail decided my entire IP block was barred from sending emails to them. Happened several times, I always had to wait a few days for my emails to stop bouncing.
In my experience it mostly works, but it’s never reliable.
Or maybe they have 10x the experience you do, but it was different experience for reasons beyond their control. Don't over-generalize from a sample of one. That's hubris.
You either get everything right (due diligence of your provider and the IPs they're giving you, configuration of your mail server, not sending spam, monitoring your reputation) and your mail is delivered, or you don't and it isn't. That's it. Getting everything right, and knowing what you have to get right, comes with experience.
Have you tried setting up a brand new email server with a fresh IP from say, Digital Ocean or AWS?
I'd recommend a smaller, tech-savvy hosting provider that has been around for a long time, who has no tolerance of abuse on their network, who is easy to communicate with directly (no paying for 'Business Support' just to talk to a clueful person), and ideally who mostly rents dedicated servers / colo rather than cheap virtual servers.
If you set everything up right, and choose the host for your mail server carefully, and never change IP, after a fairly short time you won't have much problem with being marked as spam. No more so than with any other email host.
As is so often the case, the people that say you should never do it probably have little relevant experience, they are just repeating something they heard.