> Also, most of the issues will not be with obscure email servers, they will be with Outlook and Gmail. When you send email, you will never know if your email will be delivered to inbox, delivered to spam folder, or blackholed entirely.

In my personal experience, over five years of running my own business and personal email entirely and another decade and counting since of running PBXes that directly send voicemail messages, Gmail has only explicitly blocked me once, and that was 100% legitimate because the site the PBX was located at had an infected PC. It occasionally will filter messages to the spam folder, but that almost always means one of the users actually flagged a message.

I could send an email to myself from a bogus address via raw SMTP over Telnet to port 25 and I'd be willing to bet my week's pay that Gmail delivers it. They might mark it as spam, but they deliver reliably.

Microsoft on the other hand is a massive pain in my ass. They usually work, but about 4-5 times a year they seem to randomly pick one of my systems to treat as suspicious and block emails from. The process listed in their block message always works, but it's annoying to have to deal with regularly, especially with no rhyme or reason.

Yahoo is the worst, they straight up block everything I do with no recourse. I have one user who insists on using his Yahoo email and I basically had to tell him we don't care that Yahoo won't accept messages.

From 23 years of running my own email (and for about a dozen other domains), this is a pretty good summary.

One domain has a small mailing list of about 200 users. The @yahoo.com addresses (and related, Yahoo-handled domains like old @att and @sbcglobal domains) are often just dropped.

Gmail addresses frequently go to spam, especially newly-added domains. I have found that having multiple @gmail users mark them as “not spam” tends to help, but not always.

My email server has had the same IP address for over 18 yrs, fwiw.

My emailserver runs at linode. It delivered fine for some 10 years, then I started getting blackholed (at Microsoft live, hotmail, 365, outlook). This IP does only mail. Server is secure, up to date and has all the DKIM/SPF nonsense.

It's impossible to reach humans that can help. It's impossible to unlist. So I'm now on my third IP, my last one running mail for 4+ years, from Digital Ocean. Only now, occasionally, does an email get through.

I'm almost 100% confident that Microsoft just blanket blackholes or punishes anything from Digital Ocean or Linode.

I share your confidence that lots of providers blanket drop mail from IP ranges owned by cheap VPS providers.

The cost of swtiching IPs within those providers is low, so spammers have already abused that, and made life worse for the rest of us as a result.

Yes, I had a first-hand experience with this. Cheap VPS was blackholed right away at Yahoo and any attempt to unblock it was not successful. It makes sense though.

Then I set up my own bare metal server and put it into a real datacenter serverhousing. I had couple of problems at the beginning and later once a few years later - problems sending to microsoft - but all those were resolved rather quickly via their forms. And I haven't had any deliverability problems since then.

The serverhousing I used is very strict regarding spam reports. I once tried to use the server also for sending newsletters (no spam lists, just emails which explicitly subscribed). Someone reported it as spam once and serverhousing contacted me soon after that they don't tolerate any spam. I explained that it was not spam and that there was unsubscribe link but I decided to not use it for any kind of marketing again. I think specialized services are better for this.

I use Fastmail and Microsoft Outlook sometimes puts mail from Fastmail in the spam folder. It's only Outlook that does this because when it happens, I ask what email server they use. Always outlook.

Do you/did you have feedback loop accounts setup for all of them? Yahoo, for example, sends a notification to sys@ (or whatever email you tell them to) when someone marks one of our emails as spam or when it thinks we're sending bulk email. They also have sender support. They did want me to set up something where we used the same connection to send email to them. I forgot

My "blacklisted" email company is Zoho because they consistently greylisted and blocked our correspondence with clients and have no one to contact to resolve email issues for senders.

Hi there! I'm contacting you from the Zoho Mail team. Please drop an email to support <AT> zohomail <DOT> com with the sending server logs & IP address. Our technical team will analyze and get back to you with a potential resolution. Thanks!

I can totally echo this(running my email forwarding service [0]) so I deal with it at high email volume too.

Outlook/Hotmail is kind of stupid in this sense: if the first time, a new domain send an email they are very likely to flagged as spam. But if the user simply reply or send to that email, the moving forward hotmail won't flagged it. In other word, hotmail rely stringly

- is this domain new? - is this ip address start to send email new? - if I interact with this sender in the past

Seems thing with icloud and their proofpoint spam filtering.

gmail in other hand, has the best delivery rate, clear error message, great rate limiting and cool down period.

gmail is very smart, if we accidently send spam(due to our failure to filter out spam), gmail won't blocklist the whole IP, but hotmail/icloud does.

And the process to delist from them homail/icloud are just absurb. I'm unable to contact their team.

---

[0] https://hanami.run

Hi Wolrah

I'm starting a new company where email connection to my user base is essential. From time to time I also need to sen all users certain notifications which looks kinda like marketing email but isn't.

Could you contact me as I would like to ask your opinion on the configuration that I'm trying to set up. I would also like to discuss your experiences.

If isn't too much, you can reach me by email at [email protected].

I'm not sure if you know about this, but in case not --

sendgrid is dirt cheap for transactional emails. And there's tons of offerings for marketing emails; I'm partial either to sendinblue or ses depending on your use case.

There really isn't much reason to roll your own, particularly with ses.

Rather than depend on a single person's experience about running email systems, you might check out the Mailop mail operator's list:

https://www.mailop.org/

> Occasionally, obscure email servers will block emails sent through us

The worst offender is Microsoft and their email services (hotmail.com/live.com/outlook.com/etc.).

They reject all emails from my server, because it's on DigitalOcean. They are ignoring all requests to unblock my IP address.

They don't mark it as spam, they just outright block emails, so you can't even workaround it by checking the spam folder.

> They don't mark it as spam, they just outright block emails, so you can't even workaround it by checking the spam folder.

And there’s no reason for them to do that IMO. They should dump it into the quarantine as high confidence spam. I wonder if the drop shows up in the mail flow logs.

I deal with a lot of small businesses and more than once I’ve seen bad spam filtering or silent drops cost people money (upset customers). The spam filters I can understand. The silently dropped messages should get them a class action lawsuit.

There is a reason - DigitalOcean. A more appropriate name would be DigitalCesspool. They allow anything, so long as you pay them. I'd block email from their networks, but there are already plenty of ways that spam servers on their networks screw up (bad EHLO name, broken reverse DNS, SPF for all IP addresses), so that email is not accepted anyway.

Microsoft sucks for rejecting email without recourse, but you're on DigitalOcean, so can you really blame them?

I’m not on digital ocean and I don’t know anyone who is. I’m on the receiving end using office 365. Problems I see are for mail coming from shared hosts and isps.

IMHO there’s never a reason for Microsoft or Google to drop mail silently if it’s deliverable. Let me decide if I want to do that, but don’t force it on me.

Google is awful too. Accounts locked for “suspicious activity” stop accepting incoming mail. I’ve seen people lose a day of email over a weekend before noticing ta false positive account lock.

What it really comes down to for me is that I don’t think either of them could act that negligently in a fairer market with better competition.

Things will only change if enough paying customers complain loudly about it.

You think Microsoft blocking you is bad? They ACCEPT mail and drop it afterwards.

Pretty close! They accept the mail returning a "250 OK"ish code, but they delay it for days (or even months) not respecting the SMTP protocol. I actually recevied the emails on Outlook after many many days I have sent them :/

Gmail seems good, I never had issues with Gmail.

No, not close, it's exactly what they do.

Delays are fine by the protocol, it's not a violation, it's not IM.

My emails from my domains to my Outlook account were going straight to spam.

I tried marking them as not spam, to see if it would learn. A dozen later--no change.

I tried replying to them, to see if it could figure out that if I'm actually regularly corresponding with someone they should not be marked as spam. No change.

I tried whitelisting the domain. No change.

I have the same problem, but they fixed it. I opened a issue and a kindly guy did some checks and then removed the ban.

Was it recent? I've had no issues whitelisting my IP in the past, but all my requests in the last 6 months have been ignored.

What's Microsoft's rationale ? Has an explanation ever been given ?

I've tried reaching Microsoft by phone, email, automated tools for email administrators, and their web forms for requesting whitelisting in email deliverability. Microsoft didn't provide any rational why they are filtering email from small email providers like this, nor did they provide any opportunity to mitigate their filtering.

Probably it's more convenient for them to trust X big players rather than checking one by one. And somehow an empty spam folder looks better than a full one, as it gives the user the feeling that the service itself doesn't receive spam or does a very good job at preventing it.

That's just the thing: "doing a good job preventing spam" means throwing a lot of stuff in the spam folder. I think everyone would agree that silently dropping emails is bad, even if you're really sure it's spam.

I agree, it's just that you're not the average user. I am also surprised to see the spam folder surprisingly empty all the time, almost as if my email addresses were unknown to the world (which I am pretty sure they are not).

The average user probably sees this full spam folder as something he/she is doing bad, or the service itself is bad.

It's not just google or microsoft. I think most email providers do this, I might be wrong though.

You can create anonymous servers on DO with go-betweens like Bithost, allowing you to pay with cryptocurrency. This is a pretty easy solution for people with bad intentions (bots, scraping, black hat ...)

I run my own VPN on DO and almost always get hit by ReCaptcha. Even Google.com will put up a "we've noticed some unusual traffic from" page. Some sites like Zalando completely block me with just a plaintext error page.

/me looks at his work gmail account that regularly spam filters emails from github, and emails from his work google apps domain that pass DMARC, DKIM and SPF.

Nobody can send emails to Gmail with any certainty. Not even Google.

It's bad enough that Google Drive/Docs notifications are treated as "external senders" so you have to open up your internal Google Groups to the world; then Google Groups flags them as spam anyway!

I'm guessing spammers may use Google Docs comments as a platform, but have never seen actual unsolicited Docs comments.

In my experience, it is more they "share" spam documents with you.

Whenever I'm expecting YouTube emails (pass reset or whatever) I check gmail's spam first haha. At least they don't play favourites I guess.

I moved my custom email domain from gmail to NameCheap’s Private Email about a year ago. I wanted to free myself from the possibility that Google might arbitrarily ban me one day, as we’ve seen happen to people for various Google products for no reason and with no appeal.

Biggest mistake of my life.

Just as you say, I have no idea if my emails get through. The number of times I’ve had people tell me they found my email in a spam folder or just not at all is unacceptable. And you’re right, this is most prominent on outlook and gmail users but not exclusive to them.

How would you like your accountant or lawyer to miss your emails? It happens to me all the time now. These are not emails with cat pics.

I’m scared to move back, fearing an IMAP migration will cause me to lose a year of emails sent and received.

I don’t know what to do.

I think all you really need is to own your domain.

If you manage your domain and point your mx at google, and some ai decides your drive contents look bad or some non-official youtube app violates some api access rule and they kill your whole account, you just point your at any other service provider.

That will be an annoying few days, but you don't lose anything too important. Maybe for a day or so you can't receive password emails, but you will, just after the switch shakes out.

In the mean time, you're enjoying the convenience of a big mail service, and playing the odds that it's probably not going to happen to you, and it's ok to live with that risk because if it happens, it's just an inconvenience not a disaster.

As for losing old emails, I use thunderbird on my laptop, and that ends up getting a copy of all my gmail even though I also read those same emails on my phone. If google kills my account, I still have all my old emails even without my own domain.

I don't know if google even offers domain regisration to individuals, but if they do, I just wouldn't use them for the registrar. All other services can tolerate that risk as long as you use someone else as registrar (namecheap is good).

Thanks for the reply. I thought IMAP servers keep the messages on the server, not on the client. Is that incorrect?

The client has download a copy already and wont sync back what it is already download. You can look into tools like imapsync https://github.com/imapsync/imapsync to copy data from two imap account

IMAP synchronizes with the server, but keeps a local copy. Compare to POP, which moves the mail off the server entirely.

> IMAP synchronizes with the server, but keeps a local copy.

The local copy is client dependent. Mail.app on your iPhone will not keep a local copy of all your emails if you have a large number of emails spanning a long period of time. Other clients may vary.

Don't go with namecheap, the biggest hive of scammers and spammers on the planet. Move your private domain to a reputable provider, like Fastmail.

I've had my main personal domain with them for a decade and never had a problem.

Thanks for the reply. Fastmail is indeed nice (I've used them before but not hosting my own domain). But at this point, I'm not willing to risk moving to another player besides MS or Google.

same case with me. I've been using Fastmail for 2 years, but moved to Private Email 2 months ago, and currently using Office365's Exchange.

After 2 occurrences of my mail goes into spam folder, I became paranoid and always BCC my gmail account for every important email.

In the end I say "what the hell" and subscribed to O365. Google Workspace is too expensive for me.

> you will have constant issues getting mail delivered.

I run my own obscure mail server and have not had an issue in years, except for ATT[1]. I'm super low volume - I run mail for about 20 humans, a few mailing lists, and some other automation that does things with email.

I do things correctly - I mean, I run nearly the same stack/configuration (scaled way down) as I do professionally for my employer, which delivers 8 digits of messages a day all over.

But that doesn't explain why I don't have problems and others do - lots of people get this right and still have problems. And I think that comes down to the fact that mine has been on the net since the mid-90s. So, unfortunately, I suspect the answer to a lot of these problems is 'exist for long time' - until your MX is associated with a bias towards "not spam" in everyone's filters.

I can think of ways for existing mail servers to vouch for new ones, but I don't think there's any reason for established ones to want to do that, so it wouldn't work. I'm not sure what the answer is.

[1] And after many, many years of attempting to resolve it, they can bite me. I got annoyed enough that they get a custom bounce from me now on the rare occasions someone tries to spam me through them.

I've been operating my own mail server on the same IP for the last 10 years, so not as long as yours and I also don't have problems. I also disagree with one of the parent comments that you will never know what happens to email: Gmail supports DMARC, and will give you stats and tell you what happens. You can also sign up to various sender services and domain whitelist sites and so on. I've also seen mail providers do fun things like 'permanently defer' email as a means of blocking and this actually notifies you, albeit in a disruptive way, by leaving the email permanently in your outgoing queue.

I do agree however IP reputation matters. If you share an IP with other hosters, or your IP has been recycled from a bad reputation IP, or you have a residential IP or have found your way onto any of the email spam lists recently, or been reported for abuse or marked as spam enough times by freemail service users, you may have trouble getting to 'inbox' and eventually even getting delivered. Reporting systems like DMARC should let you know, however, before it becomes serious.

I don't think running your own email is a high maintenance activity that some suggest (especially for small servers) but it is definitely more effort than outsourcing it to someone else. I'm glad services like this exist, if only to prevent email becoming a Microsoft-Google walled garden.

> And I think that comes down to the fact that mine has been on the net since the mid-90s

There are other factors like IP reputation of the email server sending the email. And if you’re using a hosted service, there’s no way to control those IP addresses.

Marketing emails are actually against the terms of service, although there are grey areas- like a user personally reaches out to offer people a service, and some occasionally end up marked as spam. If the rate is low enough, it might be acceptable.

I'll make a note to make that language stronger.

My personal email is on a .xyz domain and I used to use Gandi webmail as it was free with the domain name.

When I was finding a new job last year, every time sent recruiters an email with my resume (attached as a pdf), I had to ring them afterwards to check if they had actually received it.

I ended up switching to Google Workspace, just so that my emails actually had a chance of not landing in the recipients spam folder.

This isn't actually Gandi fault.

The amount of spam come from .xyz .top .cam etc are too hight and sysadmin usually factor domain tld into spam scoring.

so using a .com may increase your chance hitting inbox actually.

I've run into this same issue communicating with potential employers via GANDI mail--more often then not when I make a followup call my email is in their SPAM folder.

My sister-in-law's employer's mail service (k-12 school district) ALWAYS bounces back email from myself and my wife, and their sysadmin always says "Nope. Not a problem on my end."

Email deliverability is monopolised among the big players. Running your own email server and actually keeping your emails outside spam folders is more than one person's full time job. Partly this is not the big players fault but to say they don't benefit from it would be a lie.

> I have a few years of experience running my own email server and I can tell you this is a major pain in the ass. When you send your email via a small email server (a server that sends a low volume of email), you will have constant issues getting mail delivered. Also, most of the issues will not be with obscure email servers, they will be with Outlook and Gmail. When you send email, you will never know if your email will be delivered to inbox, delivered to spam folder, or blackholed entirely.

I tend to disagree, due to my experiences with running a private mail server [1]. I've had one exactly one issue with Outlook/Hotmail servers over the years. My server is quite low traffic, but it's been delivering mail reliably. How do I know? When I send mails, there's usually follow-ups.

[1] https://jschumacher.info/2021/05/running-a-private-mail-serv...

> My server is quite low traffic, but it's been delivering mail reliably. How do I know? When I send mails, there's usually follow-ups.

And when there's no follow-ups, you just assume your mail has been delivered? I recommend actually measuring your deliverability before making claims about it.

Well either there's a bounce, or some other indicator in the postfix logs (collected by filebeat, with alertings in kibana).

If I'd still be getting blackholed with a "250 delivered" in my logs, then screw it.

I would even argue people hosting their small selfmanaged mail servers helps preventing monopolization, eg. "only allow mail from google".

> Well either there's a bounce, or some other indicator in the postfix logs (collected by filebeat, with alertings in kibana).

The most common type of problem is that mail is delivered but placed into the spam folder. You will not be notified of this with a bounce or anything in your postfix logs.

> If I'd still be getting blackholed with a "250 delivered" in my logs, then screw it.

This is another type of problem I've encountered. It's rarer than the spam folder, but it happens.

This is my experience too with a private server, but I did need SPF, DKIM and Dmarc. These 3 things do require some expertise, but running a mail server already requires that. So it just needs extra time :)

Nope, setting up SPF, DKIM and DMARC is not enough to get your email delivered. I wrote about my experience more here: https://www.attejuvonen.fi/dont-send-email-from-your-own-ser...

The irony is that most of the spam I get these days comes from Amazon's AWS SMTP service.

More than just SPF, DKIM and DMARC; mails still get silently discarded if you come from an IP address with a poor reputation. For this reason most of the large VM hosting providers will work but not give great results.

I ran a kinda similar platform years ago I and I fully agree. It was a constant fight against blacklists, random spam filtering and weird attacks on my server looking for postfix missconfigurations.

If I learned anything in that time is that email is not simple, and should be run by a qualified dedicated team, because there are so many pitfalls

> When you send your email via a small email server (a server that sends a low volume of email), you will have constant issues getting mail delivered.

That’s hyperbole. I‘ve been running my personal mail server on a VPS for the past 20 years and never had any delivery issues. It may depend on the hosting provider you use, and nowadays you probably need to have DKIM etc., but there are a lot of people running their own mail server without issues.

https://www.mail-tester.com/ is a useful tool for testing.

Gmail is hit or miss, even when dkim/spf/dmarc are setup correctly. It may work 70% of the time, but it becomes a constant guessing game if you inboxed or went to spam. If you're using DO/Vultr/Linode, there may be issues with noisy neighbor IPs, but it still sucks that you may get penalized for no fault of your own. It seems like there is no incentive for Gmail to play friendly with small mail servers.

> https://www.mail-tester.com/ is a useful tool for testing.

This is great! Thank you!. I scored 2.7 out of 10. I'm using NameCheap's Private Email service. What a waste!

> I‘ve been running my personal mail server on a VPS for the past 20 years and never had any delivery issues.

Age may be a factor. I ran email for my domain on my own server for a long time too and generally had no delivery issues.

When I got another domain a few years ago and ran it on the same server it had a lot of delivery issues.

> Age may be a factor.

Probably. Although I switched both IP and domain a couple of years ago (not at the same time) without noticeable issues.

> That’s hyperbole. I‘ve been running my personal mail server on a VPS for the past 20 years and never had any delivery issues.

No email server in the world can deliver 100% (not even Gmail). Here you are claiming to have done that for 20 years straight. Well, it's not true. In fact, I'll venture as far as to guess that you've never even measured your deliverability.

I’m saying that for all practical purposes I never had an issue. That is, I don’t recollect any case where it turned out that someone didn’t get my email. That’s not to say that it didn’t ever happen in cases that went unnoticed by me, but if so it was rare enough to be a non-issue.