I remember when people thought spam was single handedly going to make email unusable.
Basically you don't control who gets to send you mails. You're entirely at Google's uncompromising and unreachable mercy.
DKIM is signing your email with the public key in DNS.
SPF is whitelisting which mail servers send mail from your domain
DMARC is how receiving ends should report how your mail is delivered (or not).
Unless you're actively spamming (ahem, "marketing") to large volumes of users, you should have no trouble at all getting mail delivered to users.
If you are doing mass email marketing, stick with your core competence and let someone else handle the mail servers.
After fifteen years running my own mail systems, I have finally given up. Ever more outgoing mail was getting lost, despite everything being up to date and done by the book. The arch villains, over and over, were Gmail and the MS thing - Hotmail, outlook.com, Live, whatever they call it.
Same here. I've been hosting my own e-mail for two decades (as of this year) and I gave up earlier this year. My spouse's e-mail to friends who use GMail--something like 80% of them--would, every couple of months, disappear into the recipient's spam folder or sometimes just disappear entirely. My logs would show a successful delivery but the recipient couldn't find the message anywhere. Occasionally messages would start going into junk in the middle of a conversation.
Nothing I changed would permanently fix it. I'd send from a different IP in my block (something.something.194 instead of something.something.193) and mail would flow again...for a couple of months and then stop. My domain registration predates Google's existence by 18 months. I've been sending from the same IP subnet for six years. My colo provider is actually so small that I'm the only customer left on my "neighborhood" /24 (they've recently asked me if I want the entire block routed to me for the lulz). I have SPF, DKIM, and DMARC all set up and they've not changed for at least two years. The amount of mail outbound from my server is (was?) so little that I could tail the outbound mail log and keep up with each message being sent.
None of that mattered. I even signed up for Google's Postmaster Tools but, and this really cooked my goose, I send so little e-mail that I don't qualify to show up on their reports. I tried contacting Google but hahahahahahahahahahahaha--wheeze--hahahahahahahaha.
No other recipient had a problem with me. Messages were successfully delivered to Microsoft and Yahoo just fine. Only Google had a problem with my e-mail.
I finally caved. Google has SO MUCH of the e-mail market that unreachability to them is a catastrophe...and they know it. I don't have the leverage to make them play nice.
Thankfully, Fastmail does so I've changed to them. For the first time since February 1997 my MX doesn't point to my own server and that makes me a little sad.
Here's some background https://wiki.list.org/DEV/DKIM
However, even that can only carry you so far. Gmail is screwed up in how they handle reputation, especially with regard to IPv6. The exact same message delivered to Gmail via IPv6 will get refused versus sending it via IPv4 will work fine.
This problem has existed for years. There are even standard hacks for postfix to cause all mail addressed to Gmail to be sent via IPv4 only, just to avoid this problem.
And any domain hosted with GAFYD is likewise screwed.
You can't know what your false positive stats are, because you don't have access to all the remote SMTP spools or logs on the 'net. All you can know for sure is the false positives that you find out about. But you literally have no way to tell how big the iceberg is under the water.
Aren't DMARC reports supposed to tell you that, especially ruf?
I only ask because I don't recall any problems over the years with one to one emails either to my gmail or from me to other people's gmail accounts.
There is a reason. It makes people switch from own email servers to gmail.
Do you have an idea what's triggering the false positives – keywords, IP reputation, etc?
The overall rate relative to my total e-mail volume is quite low, but it's enough that I have to check far more often than I'd like.
It wouldn't be a problem if I had a reliable way of whitelisting addresses, but none of the methods seems to be consistently reliable.
Note: We recommend that you do not mark your Spam/Junk Mail folder to
automatically learn "As spam". This can create a false positive feedback loop.
Imagine an email is incorrectly classified as spam, put in your Spam/Junk Mail
folder, and then learned as spam. That means future emails that aren't spam are
now more likely to be incorrectly marked as spam, sent to your Spam/Junk Mail
folder, and learned as spam. Only mark folders to learn "As spam" if they're
folders you manually move email to.
Now on top of that say you're using a MUA and its own spam filter runs atop of that, and decides to move a couple of legit stuff to Gmail's spam folder, triggering the server-side spam learning process. Surefire way to screw the whole thing up (happened).[0]: https://www.fastmail.com/help/receive/stopspam.html?u=b83140...
I've found the never mark as spam filter option to be reliable – amusingly I've also had the problem with Gmail flagging the Google Groups spam reports as spam, and this halted that.
So I have to constantly login to their site only to check the spam-folder (talk about convenience). I've setup rules to try to do my best to NEVER send anything to the spam folder but even with catch-all rules gmail still think they know better than me.
Their support for real (external) mail clients is also quite bad, I don't see why so many people in tech use gmail.
One other thing that I kinda sorta like in GMail is labels (as opposed to folders). As I understand, it's not a popular feature, because there's no reliable way to map it to IMAP? So anything that assumes that IMAP (rather than a custom client or web app) will be used as a mail reader, will probably only provide folders.
I have absolutely no problem with Fastmail's spam filtering. Maybe Gmail changed things since I switched, around 3-4 years ago, but with both, I used to get 0-1 spam emails per month, though I haven't gotten any on FM in the last few months.
While it would be quite straightforward to clone an almost pixel-perfect version of Gmail, there is something in hacker culture that makes this kind of project unattractive. I don't really understand why.
Instead what we end up with when someone starts a new mail app is some half-cocked design that makes sense for that particular user, who is always an engineer, and the result is suboptimal for pretty much anyone else.
So the project fizzles and dies before it even gets started. This happens over and over again even for larger and more visible projects like Mailpile
Gmail is _fast_. The browser doesn't reload when you navigate around and when you click on something it gets loaded very quickly. You can't just put a shiny UI in front of an IMAP server and get this sort of functionality, you have to write your own mail server with a rather intense database backing it. I have 70,000 emails spanning a decade (mostly useless) and searching is a breeze. Everything works fine and quick with a huge volume of data. When I send messages they're actually sent in a second or two, when I click next message the next message appears in a fraction of a second. The UI is the frosting on the cake. It's critical that it's very fast, but the underlying mechanics have to be very fast too, and that's not just a manner of copying pixels.
It is fast when a huge amount of JS gets loaded and latency times are low. But with high latency? A horrible experience and if it even works I'm grateful.
It is pitifully slow when on a limited connection. And the contortionist-like procedures needed to log in from an unfamiliar IP don't make it slow, just largely non responsive.
Have you tried FastMail? They use Cyrus...
> you have to write your own mail server
... but I concede they (FastMail) do sponsor most of Cyrus's development.
I think you deleted the last "."
That's what I thought, until I discovered Fastmail. I'm now a happy family account holder, enjoying great support from human beings, and happy that my children won't be data mined for advertising (at least, not in this particular case).
I'm not kidding about the support either. One of their developers was on Github helping out with an issue on an open source tool that, in the end, proved to be a problem with their implementation of a protocol. Fix was in Fastmail staging a week later, and in prod a few weeks after that.
>Unlike Google’s consumer offerings, which may show ads, we do not collect, scan or use your G Suite data for advertising purposes and do not display ads in G Suite, Education, or Government core services. We use your data to provide the G Suite services, and for system support, such as spam filtering, virus detection, spell-checking, capacity planning, traffic routing, and the ability to search for emails and files within an individual account.
Sure. I was talking about their "consumer" offerings though.
I agree it's unlikely, I'm just saying your quote leaves several possibilities open in unkind interpretations.
Oh yeah, and MX records take some time to update within popular services, so some gmail users might be able to reach you while others will get a user not found error. If you think about it, it makes sense, but it drove me nuts the first time.
Zoho is sorta free and good for BYO domain-based basic web/email.
custom bundled ldap server with lots of java custom goop. Not fun to support at scale
There _are_ open source alternatives, but they aren't as good, that's all there is to it. Setting up a mail server is hard, not because it needs to be but because the software requires eldritch incantations to make it work properly (there are too many things to keep track of in order to A) get email delivered at all and B) not become a spam relay)
It's possible to be something in between the "Just Works" walled garden that gives you no options or control and the archaic wizardry which requires a 2 foot grey beard to understand.