News for hackers. Stuff that matters.
Re:Maddy: Composable all-in-one mail server in Go by optimalsolver on Saturday November 21, 2020 @09:26PM (#25174376)
When I last looked into setting up my own email server, I read that your messages ending up in the spam folder was almost inevitable if the receiver is using a major host (Gmail, iCloud etc.)

Is that still an issue?

Re:Maddy: Composable all-in-one mail server in Go by thaumaturgy on Saturday November 21, 2020 @10:02PM (#25174583)
It's hit and miss. I've been running hosted mail services for years, both for myself and for a handful of other people and businesses.

SPF and DKIM are pretty much required now, as are TLS/SSL. From there, it turns into a dice roll. Gmail is terrible about this; they have a totally opaque and very frustrating engine that sometimes filters messages into a junk folder and sometimes doesn't. Outlook.com uses a somewhat more traditional internal RBL, but they are happy to block network segments and they don't offer any way to query their blacklist or request removal from it, so you could do everything right but a neighboring VPS or IP will get you blacklisted anyway. Comcast will simply accept delivery of messages and then disappear them depending on an arrangement of the stars that I haven't quite figured out yet. And those old sbcglobal/Yahoo services users... just, uggghh.

A popular solution is to rely on a third-party service to handle your outbounds. Sendgrid specifically is really bad, they carry way too much junk traffic, so don't go with them if you decide to try this out.

I have a post banging around in my head that's titled "Email is fractally broken", and getting outbound non-spam messages to reliably land in other people's inboxes would be a significant part of that writeup.

Re:Maddy: Composable all-in-one mail server in Go by fastball on Sunday November 22, 2020 @09:35AM (#25177736)
Anecdotally I have to disagree about Sendgrid. Have had no issues with them so far.
Re:Maddy: Composable all-in-one mail server in Go by thaumaturgy on Sunday November 22, 2020 @11:51AM (#25178855)
I happen to be running a test with them this week to see what deliverability looks like on their end. So far only 79% of their messages have been accepted for delivery by remote service providers. These are non-spam, non-transactional, typical business correspondence messages.

In their activity feed right now, SendGrid traffic is being blocked by comcast, GoDaddy, iCloud, and a cornucopia of smaller services.

Here's one of SendGrid's outbound IPs, listed on multiple RBLs used by other services: https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3a167....

On the receiving end, I've had to deal with waves of spam from SendGrid for years, and it's especially difficult because they carry a mix of spam and legitimate traffic, so blocking them causes complaints and not blocking them also causes complaints.

Re:Maddy: Composable all-in-one mail server in Go by edoceo on Sunday November 22, 2020 @03:01AM (#25175819)
Please post it when you do
Re:Maddy: Composable all-in-one mail server in Go by zertrin on Saturday November 21, 2020 @11:01PM (#25174829)
I have been self hosting my emails for many years on cheap dedicated servers (previously kimsufi by OVH and now Hetzner).

Never had any issues with sending mail. But I've been careful with the initial setup and have all the expected requirements covered : no open relay, SPF, DKIM, reverse ip, TLS,...

Been using the tutorial at workaround.org as basis for building the stack (postfix, dovecot, rspamd,...) and, while the initial learning curve was high, once its been set up, its been a really very stable setup requiring practically no maintenance whatsoever.

Not saying that's for everyone, just saying it works for me.

I guess I was lucky at the dice roll mentioned by other comments.

Ah yeah, and testing your outgoing emails using a service like mail-tester.com is very very useful. I aimed for and reached a perfect score 10/10.

Re:Maddy: Composable all-in-one mail server in Go by 7kay on Saturday November 21, 2020 @09:30PM (#25174400)
I run my own mail server and have never experienced such issues. The important thing is that your mail server is not an open relay and you set up DKIM and SPF. Oh, and never run a mail server with an ip originating in a residential area, you'll get blacklisted almost instantly by virtually any mail provider.
Re:Maddy: Composable all-in-one mail server in Go by yokem55 on Saturday November 21, 2020 @10:06PM (#25174603)
It isn't even necessarily 'residential' IP's. It's IP's that don't have a valid reverse dns, preferably one that is also forward resolvable. Most ISP's won't let you set your reverse DNS on 'residential' connections, so it ends up being a blocker. Now you could set up a vpn tunnel to a vps provider that lets you set your reverse dns, and then things get a bit easier.

So prereqs - 1) Valid reverse DNS on your sending IP. Preferably with a hostname that is also forward resolvable. 2) SPF Records 3) DKIM

Re:Maddy: Composable all-in-one mail server in Go by 7kay on Saturday November 21, 2020 @11:02PM (#25174836)
You're right, that's the basic ingredient #1 I forgot to mention. Functional rDNS is essential. You don't want to have your mail server running at home anyhow. A friend of mine had a janky setup at home until I convinced him it's a bad idea.
Re:Maddy: Composable all-in-one mail server in Go by krageon on Monday November 23, 2020 @05:27AM (#25185318)
> You don't want to have your mail server running at home anyhow

I do in fact want this. It may not be possible, but I definitely want it to be.

Re:Maddy: Composable all-in-one mail server in Go by piaste on Sunday November 22, 2020 @01:32AM (#25175452)
Anecdotally, I have a valid reverse DNS on my home ISP-provided static IP, and when I tried setting up a mail server I got blacklisted on the spot.

(I eventually went with a Tutanota account, fwiw)

Re:Maddy: Composable all-in-one mail server in Go by jeffbee on Saturday November 21, 2020 @11:17PM (#25174889)
These anecdotes are heartwarming but in the end useless as advice. If your mail is getting through it only means that there are no spammers on your network, i.e. you are lucky. Having an abusive mailer on your network (IP subnet, ASN, or even sharing your registrar) could happen at any time, with unknown impact to your IP reputation and deliverability of your mail.
Re:Maddy: Composable all-in-one mail server in Go by codegladiator on Saturday November 21, 2020 @11:56PM (#25175045)
I feel there is a lot of fear mongering around deliverability of emails all over the Internet which is helping the big companies to centralise a thing like email.

I have had my own email server sending emails which were not marked as spam. also I have had cases where email from Gmail addresses/ips would be actually spam.

I urge everyone to start using their own mail servers so the ecosystem around self hosting emails becomes super smooth.

why do receivers still rely on ip addresses when we have dkim and spf?

Re:Maddy: Composable all-in-one mail server in Go by jeffbee on Sunday November 22, 2020 @02:37AM (#25175730)
If it wasn't an effective feature for classifying spam, then people wouldn't use it. But, in reality, it's incredibly effective because the only people who want to use residential IPs and rent-a-server IPs for running mailers are criminals and a much smaller population of dorks.
Re:Maddy: Composable all-in-one mail server in Go by codegladiator on Sunday November 22, 2020 @08:08AM (#25177150)
I agree lists are definitely effective but its a cat and mouse game sort of with a lot of overhead. We can still use ip lists for accepting/rejecting mail but that should be the lowest priority check with very less weight.

Lists are like the DRM kind of tech, where the genuine user has the real headaches (pay a service to filter my mail, cant self host etc) while the spams are still flowing through.

Re:Maddy: Composable all-in-one mail server in Go by petre on Sunday November 22, 2020 @02:21AM (#25175668)
T-mobile also requires that you match your forward and reverse DNS and run a webserver with a posting address on the domain in order to whitelist your mail server.
Re:Maddy: Composable all-in-one mail server in Go by LeonM on Sunday November 22, 2020 @05:46AM (#25176458)
Yes and no. It certainly is possible to get good deliverability, but you gotta play by the rules (which is: all the RFCs). But when you do, there are also a lot of poorly configured email servers out there that don't comply to the RFCs at all, so email services need some tolerance to processing email from badly configured sources. It is almost always a compromise.

It is easy to get an email server running initially, but hard to get all the auxiliaries (SPF, DKIM, DMARC, MTA-STS, TLSRPT, BIMI, etc) set up correctly. It's even harder to debug deliverability issues, since there is 0 feedback on why your email is not being delivered. To a point where it inspired me to build an email hardening monitoring/validation service [0].

At the risk of being downvoted by people who are frustrated by email deliverability (and it is frustrating, I know): when you have email deliverability issues, it is almost always because you did something wrong on your end. Remember that false positives in spam detection hurts the $EVIL_BIG_CORP receiver as much as it does to you as a sender.

[0] https://www.mailhardener.com

Re:Maddy: Composable all-in-one mail server in Go by mrobot on Sunday November 22, 2020 @10:19AM (#25178063)
Thanks for the tool, i just used it to increase my grade.
Re:Maddy: Composable all-in-one mail server in Go by ViViDboarder on Saturday November 21, 2020 @11:36PM (#25174960)
As others have said, it depends. I’m running mine on Scaleway and it’s been working great for years now.

However, when I set up my first email server most emails were delivered just fine, however emails to a Microsoft domain (hotmail, outlook, etc) all were marked as Spam.

Then I switched to a new IP and since then everything has been great.

Re:Maddy: Composable all-in-one mail server in Go by barryp on Saturday November 21, 2020 @09:31PM (#25174412)
I’ve been able to send to gmail and iCloud ok, but outlook/office365/hotmail has been a no-go, even after going though their self-service system to open up your IP address.
Re:Maddy: Composable all-in-one mail server in Go by sneak on Sunday November 22, 2020 @12:31AM (#25175192)
It mostly depends on where you host.