There was a time my home IP on Comcast was on some blacklist with good. Every video was run through captcha. Searches too.
Comcast was useless to give me an IP in a new block.
What solved it? I signed up for gsuite free at the time and moved my email (from a colo) from “on prem” to them. Suddenly my home IP that’s used to access their services is cherry and no longer suspicious.
Note: that doesn’t seem to work anymore though. I regularly proxy some traffic through a linode and google does the same thing. Everything behind a captcha that’s stupid difficult to clearly 60% of the time. I hate traffic lights.
- Make a gmail account just for your email server, which forwards anything incoming to your host in case someone emails it directly (you'll be able to discover this gmail address if you dig through delivered emails' headers but it won't be in the From: field)
- Let Gmail authenticate with your SMTP server in Gmail's advanced options, and make sure the options are checked to retain the From: headers of relayed emails
- Generate an App Password and set up your mail server software to use Google's SMTP relay with the email and app password
There are other services to do this too I'm sure, but I'm happy with Gmail for now. And you can always transparently switch it out to another service if Google pulls something.
I plan to write a blog post on the whole process of setting up a mail server to configuring it with a gmail relay like this.
I was hosting email not just for myself but for a few other people, and that gets tricky to route through Gmail;
Gmail could change their policies at any time and give me a really bad day, potentially when I can't respond to it in a timely manner;
If routing email in this way ever triggers Google's abuse mechanisms, then potentially I'm losing access to a lot of the Google network, and while I don't use it for anything personally, sometimes I have to work with companies that do;
After spending tons of hours dealing with Gmail-related headaches despite not using Gmail myself, relying on them to get mail routed felt like a deal with the devil.
I've instead helped a bunch of people get set up with Fastmail. I love Fastmail, they're great, I miss hosting my own email but they're the next best thing. Fastmail must be handling enough traffic that they're hovering above Gmail's piss-off threshold, and really my experience with them has been extraordinarily good. Everyone I've set up over there has been happy with them too, save for one person who got told by the next IT hat-wearer that "everyone's using Google Workspace and you should be too" (and then immediately ran into a problem during setup that snowballed into a big hairy mess).
Why not if you send mail to hell? Who else do you expect to deal with?
The rule is pretty much every interaction you have in the Googleverse is easier if you just pay them something. It's pay to play internet, and yeah, it's a problem.
> It's pay to play internet, and yeah, it's a problem.
1. How much do they charge?
I'm genuinely curious. I don't self host, but use a 3rd party (fastmail). I send very few emails to people I don't know, so personally, I don't run into issues with having my email sent to spam.
2. I don't think paying in itself is the real problem. I think it's more a matter of who you pay and why you pay.
- You have to pay to register a domain name. - You have to pay to host your own server (whether your using a hosting service or hosting from your basement) - You have to pay to have gmail not mark your email as spam - ok, I'll admit, this is a little silly, but you also have to pay (via a stamp) to have USPS send letters to their recipient
3. Perhaps because so many people use and trust (whether they should or not is another question) gmail, it makes sense to pay in some scenarios? But obviously, for personal mail servers, I agree, asking to pay to play is a bit of a stretch.
If I were to set up my own mail server, that would send outbound email through Sendgrid, AWS SES, or maybe some other established and trusted sender.
Building trusts with behemoths like GMail or outlook.com is too involved an affair to be worth it in the general case, and not even through some malice on the side of the latter; it's just the reality of a medium not protected from spam.
I mean, you can't reliably send email to gmail from gmail. It's not really the origin that's the issue, it is that gmail spam classification is quite bad. Good emails from known recipients you've corresponded actively for years will continue going to spam in gmail even as you mark them not-spam for the thousandth time.
So yes, sending to anyone @gmail.com has decents odds of ending up in spam for no reason. That's just how gmail is.
But no, sending it from your own email server doesn't make things worse. I run all my own email infrastructure and sending to gmail addresses works just as well from my server as it does from gmail itself.
There's nothing like that actually. I have e-mail server running since 2000s. When gmail appeared, everything continued to work as expected and it did for many years. Around 2018 or 2019 I noticed that my e-mails started going to spam folder. I didn't send any bulk e-mails, I was not on any RBLs, I was not compromised (I actually was very careful to block outbound SMTP from any user except the one running MTA), and of course had SPF & DKIM set up.
It looks like their spam filtering is just arbitrary and feels like done on purpose to discourage running personal mail servers (and looking at comments it works really well).
Honestly, I think some kind of campaign is needed to put them in place, like starting blocking e-mails from gmail (maybe responding with a message encouraging to switch account). I remember in the past steps like that were done, but feels like today people are more acceptable of centralization.
I don't think they explicitly are trying to quash smaller independent mail servers (personal or commercial), they just see what spam statistics show, and update blocking rules accordingly, maybe fully automatically, using ML. The fact that they also quash small-time independent competition is just a nice (for them) side effect.
[1] yeah, they might be some people trying mail servers on dynamic IP, but I'm not talking about those and Google would have some justification not categorizing them as reliable. I'm talking about using business level ISP offering with a static IP address and matching RevDNS.
Sending mail from my domain to my Outlook.com address it kept going into spam seemingly no matter how many times I did any of these things:
• Find the mail in my spam folder and mark it as not spam.
• Whitelist the sender.
• Reply to the mail.
From my POV, Yahoo pioneered this way before Gmail. 10 years ago when I was managing outgoing mail infra for a company that probably sent 100k emails per day (mostly purchase receipts and subscription notifications), I was intimately aware of deliverability characteristics for Gmail and Yahoo, which were our two biggest destinations by a large margin (if I recall, Gmail was about 60% and Yahoo was about 30%). Gmail, despite being the overwhelming majority of our traffic, was hardly any problem at all, whereas Yahoo would regularly give us the dreaded 451 response that indicated that your mail was going to be held for 2-24 hours and then randomly rejected or accepted.
BTW, if I were to run my own mail infrastructure again, I'd definitely use someone like Sendgrid with a dedicated outgoing IP with a clean reputation. It makes a very big difference. We eventually ended up with our infrastructure split across multiple IPs by function (e.g. receipt emails were the only thing ever sent from one specific IP because we wanted to ensure deliverability).
Then when your mails to Yahoo got repeatedly blocked by Yahoo and you got tired of having to repeatedly report the false positives and and get unblocked only to get blocked again a few weeks later you have the person in charge of your ad spending call up your ad sales rep at Yahoo.
You asked the ad sales rep "Why should we keep spending $X/month on Yahoo ads to try to acquire customers, when Yahoo mail keeps blocking the receipt emails, setup instruction emails, and response emails to support questions we try to send to those customers which angers those customers and they cancel (or even try chargebacks)???".
What happened then is the sales rep puts your ad manager on hold for a few moments, then comes back with someone high up in Yahoo IT management conferenced in, and explains the situation. The high IT manager puts you all on hold, and comes back in a few minutes with someone who actually deals with maintaining the spam filters conferenced in, explains the situation, and tells the maintainer to add your domain to a special whitelist of mail that is always accepted.
People here saying "it's spam, not Gmail" are being distracted from the numerous issues that independent mail services do have with Gmail.
Gmail is extremely uncooperative at accepting email from services that aren't Gmail, Comcast (sometimes), or Microsoft. You can have everything configured correctly, on an IP you've owned for years, and aggressively manage any outbound spam, and Gmail will still hate your guts and bounce your email or file it in the recipient's Junk folder.
Before Gmail got huge, email service providers typically offered an avenue for addressing false-positives in their filtering systems. Gmail really pioneered the "nah, screw you" approach to this.
Meanwhile, Gmail is itself a huge source of spam.
I (maybe perversely) loved hosting my email and email for a handful of other people. It's fun. Gmail took all the fun out of it and turned it into a seething hatred.