News for hackers. Stuff that matters.
Re:G Suite Horror Story by Someone1234 on Sunday August 05, 2018 @01:00AM (#17689770)
I'd strongly recommend against it.

In a past life I ran email services for a small business, and then a medium business, it was a full time job, just keeping us out of the spam blacklists, keeping things patched up, ecosystem changes (e.g. new DNS records every few years, new security requirements, etc) and filtering incoming spam/malware/philishing attempts.

If people want to setup email for educational reasons, go ahead, but running one over a longer period isn't wise and frankly many wouldn't put enough time and effort into it to do it well. It is absolutely not set and forget, you'll need to baby it daily indefinitely.

When people ask me what I recommend? I tell them outsource email to a major provider and use your limited time/effort elsewhere. Better rate of return by far.

Re:G Suite Horror Story by cm2187 on Sunday August 05, 2018 @02:11AM (#17689961)
I disagree with the full time job. When you run it for yourself and your family it is not that much work (if you have to support lots of people, different story). Particularly if you don't have to deal with the hardware (VM). To me the biggest objection is cost, and technical complexity to set up.

For managing spam, I found that the best solution when you own your own domain is to give a different alias to anyone who wants your email and keep a record of the mapping (I had to create a little website that does that for me). If you start receiving spam, then just delete that alias (this is why the "+" alias feature in gmail not really useful, plus it leaks the underlying email). I use Smartermail, which runs on windows, and allows you to reply to an email from the alias it was sent to. This is a low maintenance, zero spam solution.

For patching, windows patches itself. You need to update smatermail manually though, so it takes about 10 minutes every month or so. I automated the renewal of certificates with letsencrypt.

The only thing that has been a source of worry and requires to keep an eye on is failed login attempts (I scripted it but it is worth monitoring). Once I had created a test email account with a weak password, and some spambot got access to it by bruteforcing passwords in smtp. Then it started sending tens of thousands of spam emails from that account, which got me on gmail's blacklist for a month, even if the whole event lasted less than one hour (smatermail notified me of the suspected spam activity).

Re:G Suite Horror Story by mehrdadn on Sunday August 05, 2018 @03:15AM (#17690157)
How do you avoid downtime? Gmail is not going to go down out of the blue, but my own server could easily do that when my internet randomly goes out.
Re:G Suite Horror Story by cm2187 on Sunday August 05, 2018 @04:00AM (#17690276)
If you want to send emails, don't run it from home. Most mail server will increase the spam level of an email coming from a dynamic ip or where the reverse dns doesn't match the name of the server.

If you just want to receive emails, then it depends how long is your downtime. Smtp requires that the sending server makes several attempts if the first failed, so if you are only off for a few minutes to a few hours, you will get your emails, just delayed. If full day downtime then this is not a viable solution.

If you need to send emails, then you kind of have to host your server in a datacentre. A cheap VM should probably be a good solution for a first attempt. You will still get downtime occasionally (update OS, update mail software, problem with your hosting provider). A cheap solution is to set up a failover server at home since it is only to receive emails while the primary server is down, with a lax retry policy (up to a few days).

Re:G Suite Horror Story by chronid on Sunday August 05, 2018 @04:52AM (#17690405)
A lot of IP ranges from cheap VM (and bare metal) providers are included in many blacklists since those ranges were (predictably?) abused by spammers in the past. Some companies are fighting this by denying outgoing traffic to port 25 until you fill a support ticket now, but whatever is in the blacklists will (probably) stay in the blacklists. Even buying your own IPv4 range does not guarantee it was not used by spammers in the past, and you will have to fight the blacklists owners to have it removed...

If you want to be sure your mail gets to someone else, things are going to be more complicated than setting up an SMTP+IMAP server combo, and after dealing with it in the past for work I'm happily outsourcing the pain to fastmail nowadays. :)

Re:G Suite Horror Story by ocdtrekkie on Sunday August 05, 2018 @08:37AM (#17691028)
Blacklist owners are fairly reasonable. You do need to have an IP dedicated enough that your Reverse DNS can be set to your domain, and then pretty much all blacklists will honor a request to delist it or whitelist it. I've dealt with a couple of them at work.
Re:G Suite Horror Story by drdaeman on Sunday August 05, 2018 @12:29PM (#17692056)
You don't need to avoid it. Email was designed in the age of unreliable connections. It can withstand downtime just fine.

Almost every mail server retries for a few days, so you'll get your message, just late. Those few mail systems that don't retry have issues with greylisting - and unless you're fond of stories about Nigerian princes in distress, you're going to use that as the first anti-spam measure anyway.

Re:G Suite Horror Story by mehrdadn on Monday August 06, 2018 @05:32AM (#17696222)
It can only withstand so much downtime though. What if you don't have access to your server for a week? Heck, even a couple days can screw things up. So much email is time-sensitive. From transit delay notifications to message notifications and bank alerts... to actual humans that matter in your life trying to get something from you by tomorrow thinking you'll get their emails as usual. Maybe it's just me but I just don't see how "oh it's chill if my emails get randomly delayed for a day or two" would fly in today's world. Intentionally going off the grid after you've prepared for it is one thing, but having your email randomly go out without any warning in a potentially awfully inopportune time is quite something else...
Re:G Suite Horror Story by drdaeman on Monday August 06, 2018 @03:22PM (#17700820)
> What if you don't have access to your server for a week?

Then you've chosen a bad service provider(s) to host/colocate/peer with.

Actually, you should notice this situation pretty much immediately you're online - because most likely your IMAP server would be down - and every single email client I've used had started to display warnings.

If you have more complex setup with separate MTA so your IMAPd may be online while your STMP service's unreachable, then you should set up monitoring system and deliver alerts either using local delivery or any out-of-band mechanisms (like SMS).

Also, you can set up a backup MX. Or two. Or more. Email has failover/HA since forever.

Re:G Suite Horror Story by 8_hours_ago on Sunday August 05, 2018 @02:13AM (#17689972)
I highly recommend FastMail for a paid email provider. Their support staff is quick to respond, and it’s nice to be a customer instead of a product (i.e. pay for the service instead of getting it for free with ads)
Re:G Suite Horror Story by imron on Sunday August 05, 2018 @09:43AM (#17691269)
I second this. I ran my own mail server for a couple of years (a good educational experience), but after a while I figured it was far less hassle to pay someone else to deal with it.

FastMail have been an excellent email provider, and support has always been responsive and useful.

Re:G Suite Horror Story by ekovarski on Monday August 06, 2018 @01:28PM (#17699692)
FastMail is amazing; there were a few hiccups early on with the native iOS productivity apps but its been smooth sailing for years now.

If you can, definitely try to run your own mail server for the experience but start of gradually, e.g. relay outbound email through an smtp service, and slowly add the different layers such as spam analysis, filtering, etc.

There are /so/ many layers to email these days that if you try to do it all at once you'll spend way too much time on just keeping the wheels from turning.

Re:G Suite Horror Story by ulrikrasmussen on Sunday August 05, 2018 @02:11AM (#17689963)
What? I have been managing my own mail server and domain for 10+ years now, and apart from a few days spent reading up on DKIM, DMARC, SPF, etc., it has been mostly set and forget.
Re:G Suite Horror Story by nvarsj on Sunday August 05, 2018 @08:42AM (#17691052)
I did the same for many years. Until I noticed that some people just never got my emails. Mostly people with business emails - I'd send something job related, and never hear back, just assuming they weren't interested. It took me a while to figure out that some email providers (outlook.com is one example) will just outright block small email hosts, despite following all the best practices like DKIM, DMARC, SPF. It's a lost cause trying to host your own email. I've been on fastmail for the last year or so with my custom domain, and it all seems to work great.
Re:G Suite Horror Story by howard941 on Sunday August 05, 2018 @01:53PM (#17692511)
It's ironic that outlook.com blocked you when it itself is too costly to block in that you can't tell whether the message is important from a contact at say a Fortune 500 or another Nigerian scam.

Like you I ran my own stuff but possibly unlike I didn't encounter issues with recipients other than yahoo. I ran it from home from 2001 until last month and paid extra for a /29 static block from the phone company.

Re:G Suite Horror Story by Avamander on Sunday August 05, 2018 @11:33AM (#17691815)
> will just outright block small email hosts

And that's exactly why self-hosting needs to be more dominant instead of just relinquishing power to large corporations.

Re:G Suite Horror Story by jocoda on Sunday August 05, 2018 @02:25AM (#17690008)
My experience has been likewise.

Only problem I had, and still have in the short term, is that one server's IP C block has been flagged by one spam service. And even though my server's IP gets a clean bill of health from all other SPAM services, outlook.com refuses to accept mail from that server. I had them white list the IP but it reverted shortly after that.

The ISP (strato) refuses to do anything about it, and not sure if there is anything they can do, so all I can do is vote with my feet.

Otherwise, running a personal mail server is hardly any effort at all. Use Webmin for admin.

Re:G Suite Horror Story by algesten on Sunday August 05, 2018 @04:14AM (#17690309)
it's only outlook.com? you can't fix it?

i mean that's a rather large group of people you can't communicate with right there.

and it's the only one you know about.

i say this is the exact reason running your own mail server in 2018 is problematic. you just chose to not consider it a problem.

Re:G Suite Horror Story by tolien on Sunday August 05, 2018 @09:04AM (#17691133)
> i say this is the exact reason running your own mail server in 2018 is problematic.

It's a reason why allowing these fairly core services to be provided by a shrinking group of organisations is equally problematic.

Maybe one day Google will decide that they get too much spam from outlook.com (or vice versa) and cut them off too. What are you going to do about it, set up your own mail server? Vote with your feet to another provider? Start a company doing the same? Good luck with that.

Re:G Suite Horror Story by jocoda on Monday August 06, 2018 @06:32AM (#17696444)
It's not the only mail server I have running but it is the only one that has this problem. So bye bye strato.

>.. it's the only one you know about.

In the case of outlook.com, there's a delivery failure message. My yardstick is gmail.com. If the mail is getting to gmail accounts then I'm confident that it's as good as it is going to get.

Re:G Suite Horror Story by Qwertie on Sunday August 05, 2018 @02:41AM (#17690054)
I have run email servers for myself and businesses. Every single headache I have had was related to some user having a shit password or malware on there PC and then spammers send spam from their address.

My personal email server has been hassle free. I use https://mailinabox.email/ which you just run the setup script and tell it your domain name and everything is set up correctly. It sends me an email every now and then saying it needs updating and thats all.

Re:G Suite Horror Story by mcny on Sunday August 05, 2018 @09:12AM (#17691156)
How do I set up mail in a box on a sub domain? Example.com is on outlook which I cannot disrupt but I’d like to set up and programmatically access all the emails coming to anything at dev.example.com

Preferably just query what emails do I have coming in for local name at dev dot example dot com

Deliverability is not a problem because I don’t intend to send any mail. Just need a sinkhole I can query. How difficult is this?

Re:G Suite Horror Story by Qwertie on Sunday August 05, 2018 @10:40AM (#17691542)
I haven't done it but I see no reason why it would be hard. You just tell it the domain to use which should be able to include the subdomain. One thing is that usually you point the domain's nameserver at the mailinabox server and it sets up all the right records for you. If you can do that on a subdomain then it's super easy. If not you just have to manually set all the records. Mailinabox shows you everything to set if you do it this way though.
Re:G Suite Horror Story by anonytrary on Sunday August 05, 2018 @01:27AM (#17689854)
> keeping us out of the spam blacklists, keeping things patched up, ecosystem changes (e.g. new DNS records every few years, new security requirements, etc) and filtering incoming spam/malware/philishing attempts

It is so easy to forget how much noise and junk gets passed around via email when I've had Google doing it for me for all these years...

Re:G Suite Horror Story by reustle on Sunday August 05, 2018 @04:34AM (#17690353)
Does a service (api?) exist where I can run my own email server but all email is forwarded to this service and it returns a Boolean for "is spam", then my local email server files it into a spam folder automatically. This way I always keep full control and can exclude forwarding emails from my contacts list.
Re:G Suite Horror Story by pas on Sunday August 05, 2018 @05:01AM (#17690427)
rspamd is as good as it gets, you can run it locally, you need to train the filters "of course", but you can do it incrementally - as mail pours in you put spam into the spam folder [0], and rspamd (via rmilter via postfix) puts a X-Spam-Flag header into the mail, so sieve (part of dovecot) sorts if out [1].

of course, there are a bunch of cloud/hosted/SaaS versions, usually they front-proxy your email servers via MX records [2]

[0]: https://thomas-leister.de/en/mailserver-debian-stretch/#spam... [1]: https://thomas-leister.de/en/mailserver-debian-stretch/#glob... [2] https://www.everycloudtech.com/

Re:G Suite Horror Story by shirro on Sunday August 05, 2018 @01:43AM (#17689888)
I have spend a handful of days over a decade and a half researching or experimenting with my family mail server config. Given that experience can be useful professionally, even if only to understand the services you are purchasing it seems like time well spent for an individual/family to me.

It depends what your focus is within the industry and how much you like to be hands on. If you are an auto mechanic you might service your own vehicle. If you sell vehicles or drive them professionally then perhaps not. It isn't like any single approach is right or any particular role is preferred.

Re:G Suite Horror Story by lowry on Sunday August 05, 2018 @02:32AM (#17690025)
I am an enterprise programmer by day and I've been running mail servers for 20 years both for hobbies and for private use. I rarely spend more than an evening per year, mostly to keep up with new developments in that space. A few years ago I deployed DMARC. Last year I registered in postmaster tools at Google, Yahooo, Yandex and a few others. This is it.
Re:G Suite Horror Story by zAy0LfpBZLC8mAC on Sunday August 05, 2018 @01:26AM (#17689848)
> It is absolutely not set and forget, you'll need to baby it daily indefinitely.

That sounds like you were doing something very fundamentally very wrong.

I definitely don't need to touch my mail server daily, nor any of the mail servers I manage for others. It's just doing its job, security updates get installed automatically, and every few years I need to do a debian upgrade, and possibly deploy some new security/anti-spam features. In between those upgrades, it's running unattended for many months with absolutely no problems. On average, I touch my mail server maybe once a year.

Re:G Suite Horror Story by sneak on Sunday August 05, 2018 @02:04AM (#17689935)
I can second this, but it’s a full 8 hour day once per year when some spamassassin or amavis config breaks in an update... not to mention the normal sysadmin stuff of backups, verifying backups, logging, monitoring, et c. It’s a big project to run a mailserver. (Source: been doing so for 21 years.)
Re:G Suite Horror Story by zAy0LfpBZLC8mAC on Sunday August 05, 2018 @03:23AM (#17690178)
Well, you also need backups with a "cloud" solution, so that's no difference.

As for monitoring: If you only run it for yourself or a small org, really, you can do just fine without monitoring. If the mail access side breaks, you'll notice, if the inbound side breaks, you'll probably also notice soon enough. But then, setting up a minimal monit that tells you when storage runs low or the MX is down really isn't that difficult either.

So, no, it's not a big project. It's as much of a big project as owning a car, probably less so, if you consider the amount of effort/time required (assuming you have an IT background, obviously).

Re:G Suite Horror Story by godzillabrennus on Sunday August 05, 2018 @02:05AM (#17689937)
I also managed email for a living and helped transition many enterprises to cloud email solutions.

It’s quite likely that other, unauthorized folks, are touching your mail system more frequently than you with such a lax attitude.

Definitely outsource your email unless you want the experience.

If you want security it seems reasonable to use ProtonMail.com instead of larger cloud providers like Google or Microsoft.

Re:G Suite Horror Story by michaelmrose on Sunday August 05, 2018 @02:48AM (#17690082)
"It’s quite likely that other, unauthorized folks, are touching your mail system more frequently than you with such a lax attitude."

This is complete nonsense. I cannot imagine that there is anything particularly magical that requires something other than regular security updates on any server.

Manual intervention is borne of changing requirements or broken things.

Why should an email server for a handful of users require a lot of manual intervention?

Re:G Suite Horror Story by zAy0LfpBZLC8mAC on Sunday August 05, 2018 @03:35AM (#17690207)
Could you explain where that magical unauthorized access to my systems should be coming from?

Really, you sound more like someone who has read some FUD marketing material than someone who knows how to write an exploit or has any other serious IT security expertise. I am pretty sure I never had any unauthorized access to my mail server in the last ~ 20 years that I've been doing this. Obviously, my system isn't immnune from exploitation, but neither are Protonmail's, and you are massively exaggerating the risk.

Re:G Suite Horror Story by catdog on Sunday August 05, 2018 @06:10AM (#17690564)
He clearly states who likely pays for spreading this FUD…
Re:G Suite Horror Story by ms512 on Sunday August 05, 2018 @11:36AM (#17691827)
While I agree with setting up an email server not being a trivial endeavor, I managed to learn a lot and complete setup within a few hours by following one of the many available tutorials ([1], [2]) and the ongoing maintenance has been minimal.

I took the plunge about 4 years ago after being uncomfortable with the possibility of losing my main Gmail account. Ultimately, I decided that if I didn't own my email address, I was at the mercy of Gmail if anything went wrong. I came to the conclusion that losing my main email account would be more painful and time consuming than running my own server. At the time, I considered an alternative of owning my own domain and doing mail forwarding to Gmail or using a smaller email hosting provider, but decided there was value in learning more about how email worked.

[1] http://flurdy.com/docs/postfix/ [2] https://www.linode.com/docs/email/running-a-mail-server/

Re:G Suite Horror Story by setquk on Sunday August 05, 2018 @12:00PM (#17691934)
It’s ok running your mail server until yahoo black list you and there is no recourse for 6 months other than all email to yahoo will be deferred permanently.

Unfortunately I know lots of people on yahoo (still) and no amount of brow beating appears to be changing that.

Paying for FastMail did the job for me in the end. It’s the least shit solution I’ve encountered so far. Note I didn’t use the word best.

Re:G Suite Horror Story by syshum on Sunday August 05, 2018 @10:24AM (#17691462)
>>>In a past life I ran email services for a small business, and then a medium business it was a full time job, just keeping us out of the spam blacklists

Running it onPrem using the ISP the is connected to office I bet... That is a common problem with small businesses as they do not properly setup the ISP service and many SPAM lists automatically add the dynamically assigned IP ranges of most ISP has these are not setup to allow hosted services across them so by default no one expect to see a email server on that type of connection. Thus it is blocked.

Outside of that companies like MailRoute can help with routing to email traffic, and provide some level of HA for your email services.

>> ecosystem changes (e.g. new DNS records every few years, new security requirements, etc)

SPF, DKIM, DMARC are not really new, nor have they changed much in years. Thought most people still fail at getting it right. I am dealing with an issue with my bank right now where one Division is not talking to another division but they all use the same main domain for sending email so of course SPF record does not have ALL of the mail servers listed ... sigh...

>>filtering incoming spam/malware/philishing attempts.

There are many 3rd parties out there providing these types of services, I would use them. MX Gaurd Dog is probably one of more reasonably priced services but there are all kinds of other from Mimecast to Proofpoint, and many many others

>>I tell them outsource email to a major provider and use your limited time/effort elsewhere. Better rate of return by far.

I am not opposed to that persay. I also advice my clients if they are going to do that, at minimum they need a email archiving solution that is NOT with that same provider so they do not have all of their critical communications locked up in a single point of failure. For better or worse many companies live and die by email and if Email data is lost that can be DEVASTATING to a company. Having it all locked into a service you do not control is a high risk business ending decision IMO

Re:G Suite Horror Story by derefr on Sunday August 05, 2018 @03:17AM (#17690162)
> filtering incoming spam/malware/philishing attempts

Is there not just an open-source pre-trained spam-filtering model (using whatever is the state of the art for these—guessing classical Bayesian filtering on word-tokens is no longer at all what Gmail is doing)? It’d be a nice complement to the open RBLs, no?

Re:G Suite Horror Story by ivan_ah on Sunday August 05, 2018 @05:14AM (#17690453)
There are, but they are not state of the art.

There is also the benefit of the spam filter service being "live updated," e.g. if a new type of spam message comes up it will get flagged since lots of users.

Re:G Suite Horror Story by yani on Sunday August 05, 2018 @01:24AM (#17689843)
I keep hearing the same over and over from sysadmins. Is this too difficult to automate?
Re:G Suite Horror Story by jwr on Sunday August 05, 2018 @07:22AM (#17690776)
Ask an architect if it's difficult to build a house, and he'll scare you with tales of complexity. But that doesn't mean you can't build a shed yourself.

It can't be completely automated, but it isn't as difficult or time-consuming as people paint it to be (especially sysadmins!). I've been doing it for nearly 18 years now, maintenance takes perhaps 30 minutes every 2 months or so. There is more to do every 8 years or so, as Linux distributions go out of date, and your providers sometimes go out of business.

I have my suspicions as to all the FUD regarding running your own E-mail server. Secure personal E-mail distributed on many independent servers with automatic in-transit encryption is not in everyone's interest.

Re:G Suite Horror Story by zatkin on Sunday August 05, 2018 @01:38AM (#17689877)
Having a lesser known TLD (.gd) helps in my case. I get spam email once in a blue moon.
Re:G Suite Horror Story by cm2187 on Sunday August 05, 2018 @02:13AM (#17689967)
I believe most of the spam comes from your email address being leaked by a company or website. Not sure how what your TLD is has anything to do with it.
Re:G Suite Horror Story by ValentineC on Sunday August 05, 2018 @09:36AM (#17691244)
I feel like this particular issue happens most often when one uses a catchall, as spammers like to try common usernames and this setup will allow all of them to be received.

In those cases, I found that it helps to use an obscure subdomain that doesn't have a website attached to it (e.g. mailus.example.com).

Re:G Suite Horror Story by mehrdadn on Sunday August 05, 2018 @03:18AM (#17690165)
I would guess it makes it harder to recognize for website scraping if your email is left online somewhere.
Re:G Suite Horror Story by jdietrich on Sunday August 05, 2018 @09:24AM (#17691200)
A lesser-known TLD will massively increase the likelihood of your outbound emails being filtered as spam.