> It's insulting to the folks that are pouring their time replying to what they think is a user in need of technical assistance, only to eventually stop replying when it becomes obvious that he only replies with questions of morality and treats most answers as if they were platitudes specifically attempting to dismiss his efforts to communicate.
Yes, I have been bitten by this many times. Every time I see a topic like this one I try to help that person by offering technical advice, basically doing my consulting job for free for everyone to read. And many times the post gets downvoted till its dead.
Slowly I learned that people don't want a solution to their problem, they just want to vent their (understandable) frustration about $BIGCORP. But it still hurts to get downvoted when you just try to help.
People ignore the email spammers, SEO spammers, automated password crackers and blame google for the solutions they put in place to combat those things.
It's impossible to reason with them, honestly I think it's just best to ignore posts like this.
I have my mailserver configured in pretty much an optimal way, with all the special bells and whistles, have never sent spam, and never will.
Gmail refuses to deliver my mail, so please don't lump me in with malicious people and then say my concerns should just be ignored.
In the end, TFA is from a subdomain that seems to have a lot of spam, and to avoid it may mean using A commercial option, if not a specific commercial option. It sucks.. but the mail relays I mention above are relatively cheap and easy to properly setup with limited issues on the other side.
There is usually some misconfiguration somewhere that goes overlooked. Happens to the best of us :-)
What is your opinion (or others' if they'd like to chime in)? If you enjoy running your own, are there any guides in particular that you'd recommend? Does the future of email look even more prohibitively complex for self-hosting?
Following a guide to setup a basic email service is quite easy to do, but after a while the lack of proper implementation and maintenance of the standards will start to cause deliverability issues. Those are frustrating and most end up writing a long rant on why Google is evil because their email ended up in spam. When in fact, it usually is a misconfiguration, but those can be really hard to detect.
IMO there is still great satisfaction in running any internet service yourself. But, if you rely on this service to work, it just no longer makes financial sense to do so.
It takes resources to keep any internet service running correctly. Even the simple stuff can quickly get overlooked, I have seen so many self-hosted email servers that have expired certificates (like with the domain of the OP) or lack support for any of the modern security extensions such as DMARC or MTA-STS.
For most businesses my recommendation is not to run it yourself. Like you wouldn't run your own DNS service anymore. Of course it can be done, and in some cases it makes sense to run stuff yourself (usually when operating at a very large scale). But it almost always makes more financial sense to use a hosted email solution.
We haven't written a guide (yet) on setting up an email server with all the bells and whistles. But if you want to read more on all the email security standards, have a look here: https://www.mailhardener.com/kb/
For delivery, if you are really just using it for personal use a mailgun or sendgrid account can allow you to handle most delivery issues, while still handling inbound.
Which still leaves the other side, actually dealing with inbound mail issues which is just painful with either too much or too little filtering. The likes of google and MS have teams of brilliant people to handle dealing with spam, and despite some little guys caught in the middle, is still far better than I could ever do on my own.
In the end, if you want a career in IT, or find it interesting, I'd say go for it. I've often thought about building my own end to end open-source mail service aimed at ease of administration. If I was ever rich enough to not have to work, that's one of the things I'd probably do after a few months off.
I'd suggest starting with a secondary domain and using Mail-In-A-Box on a VPS or smaller Cloud host. It's a decent starting point, and there are many other options. One of my favorites is SmarterMail (commercial, windows only though). I also ran a BBS for a number of years using Synchronet, which does email/pop/smtop and even nntp for group messages. I did have it configured with SendGrid at the time for outbound for a while, which handled delivery issues for me.
You will have headaches if you take it seriously and/or use it as a primary service without using a delivery service like MailGun or SendGrid, and even then you probably will have other headaches. My point isn't really do discourage so much as let people have a more realistic understanding. The issues the OP has are real. However, there is so much junk from bad actors that the well is poison.
Please make your substantive points more thoughtfully.
What sort of config do you require from the OP? Their domain name is public and uncensored right in the top of the post. Their configuration has been cross-checked by many people so far. A lot of folks claimed that it has issues (like a `?all` in SPF, or `p=none` in DMARC, or some other nonsense that any spammer would easily get right), but those issues have been rebuked as not being significant to the issue at stake, plus, there's absolutely no confirmation that fixing these "issues" would resolve the problem, either.
Google provides a service to its users, they don't provide a service to you.
Unfortunately for you, the e-mail providers' customers (the users receiving the mail) get to decide what is an unacceptably high false positive/negative rate --- and not you. If e-mail recipients get too much SPAM or if they get mail that they want landing in their Spam folder too often, then they will switch. This provides the economic incentive for mail providers to Get Things Right.
Unfortunately, there is so much spam out there that if you are too relaxed with letting marginal e-mails (or e-mails from marginal network neighborhoods through) the false negative rate could result in so much spam in users' inbox that they will get mad, and switch.
However, it can happen that the email provider can be too sensitive, and there can be too high of a false positive rate. For example MIT recently tried relying Microsoft's spam filtering system. Way too much stuff landed in the spam filter, and while you could go to the spam filter and manually mark e-mails as "not spam" and eventually Microsoft's ML algorithms would hopefully figure it out, the false positive rate was too high and the MIT community (especially the ones with the real power, e.g., the professors) rose up with an outcry, and MIT abandoned using Microsoft's Spam filtering service.
They can be pile driving right next to the machine room, and causing disk drive errors, and Physical Plant won't care if the I/T folks complain. But if some Sloan School professors complained that it was disturbing their cogitations, it immediately stopped and Phys. Plant called a "stakeholder meeting" and it was really obvious who had the power.
All customers are equal, but some customers are more equal than others.
Even the particular people who irrationally hate something are obviously likely to also run across the good reasons for hating it.
It's not reasonable to say "criticism X doesn't count anymore because we've heard it too many times".
Pro tip: start by briefly acknowledging the frustration, then offer the technical solution/workaround.
People forget sometimes that the person on the other end of the screen isn’t a computer that is completely unmoved by silly things like emotion and frustration.
As a hobbit* , I'm actually pretty happy this whole conversation is starting to gain more traction.
My domain is always green on every test I could run (mailtester.com & all) yet some of my mails go to the junk without any warning. At least warn postmaster when you decide to block a domain because of bad neighbour/hosting!
* This specific reply of the chain is a must read and is where learned that I'm actually a mail hobbit. https://www.mail-archive.com/[email protected]/msg08958.html
This exactly. I've been a Gmail user since the day when they've required an invitation in order to signup, and I'm not quite a Google hater, either, but it's extremely disappointing to see the number of people who simply defend Google against the wrongdoing irrespective of whether or not the complainant has a valid complaint, like in this very case.
What happened to the whole notion of believing the victims? Why no matter how hard you try to explain your setup, and the things you've tried, and that all the automated tests pass, that you still get blamed to be the culprit for the issues you experience when dealing with someone who has a monopoly on the service at stake, instead of considering the possibility that maybe it's Google Mail having the monopoly on email that's an issue here.
Also, if you aren't sending a lot of mail, consider relaying outbound through mailgun or a similar service (I used to use sendgrid myself). Nearly zero issues when doing this on the lowest level paid account. That said, inbound spam was so painful to deal with I just gave it all up.
Princeses were the customres which sometimes required a lot of customer support, but paid really well because they purchased a lot of services/VM's. Hobbits were customers who basically minded their own business, and didn't guy a lot of stuff, but didn't cause any problems.
Lizardmen were customers which had possibly clueless marketing departments that would occasionally send e-mail marketing campaigns which crossed the line, but which also sent plenty of legitimate e-mails, and which were mostly trainable after the ISP/cloud provider smacked them on the nose with a rolled-up newspaper --- and Kobolds were spamvertising companies that did nothing but send spam, and were constantly switching ip addresses / VM's, and knew d*mn well that they were doing something evil, and were not trainable.
The problem is that if the ISP / cloud provider has the vast majority of their customers being Kobolds and Lizardmen, it becomes economically hard to just cut off the bad actors, because the vast majority of their customer base are bad actors. If the provider originally had a good mix of all of these customer mixes, but wasn't proactively cutting off the Kobolds and educating the Lizardmen, then their network block would start getting a bad reputation, and all of the major e-mail providers (not just GMail) would start blacklisting the netblock, or at the very least, treating any e-mail sent from that network block as a strong SPAM signal, and this would cause the other customer types to find other providers.
So if you happen to be a customer of such a network/cloud provider, you could try complaining to the provider, but they are really in between a rock and a hard place, because if you're the 1 good hobbit and all of the otheer customers for that provider are Kobolds and Lizardmen, what is the provider supposed to do? You're probably better off switching to a better provider. Unfortunately, the better provider may also cost more, but you get what you pay for.
Because it's really infuriating if someone writets to you, you put time to help them with something, and their demented (did I say it's infuriating? :D - also what kind of company can't detect something is a response, if you even quote the original email, it's ridiculous) e-mail provider trashes your response.
Sadly, I'd imagine some users don't really have a choice, e.g., an employee whose company is using Gmail from G Suite for its email, so, it'd be an uphill battle for sure.
So if someone can't be bothered to check so that I'm not wasting time responding to their trash folder, that's a good filter in the first place.
Anyway, better thing might be responding by adding a (F)AQ item publicly, and just sending a link.
The common thread among many of the bad-Google stories I've seen in the last 10 years is that Google has made a deliberate decision to not give two shits if they accidentally hurt a small minority of their users. People get messages out of the blue that essentially say "We've just wiped out your email account, we're not telling you why and there's no one you can appeal to." I don't know if this particular guy is on the level or not, but it's not credible that all of these people are secretly spammers and scammers. Occasionally you can get things fixed if you know someone at Google or your complaint goes viral, but that's your only option.
I'm sure someone at Google did some math and concluded that the ill will of a few customers costs less than maintaining a support department. That's not an excuse. I don't think we should expect Google to never make mistakes, but we should absolutely expect some avenue to correct those mistakes.
It's insulting to the folks that are pouring their time replying to what they think is a user in need of technical assistance, only to eventually stop replying when it becomes obvious that he only replies with questions of morality and treats most answers as if they were platitudes specifically attempting to dismiss his efforts to communicate.