Plus many spam blacklists have blacklisted entire IPv4 ranges as "residential IPs" (meaning even if your ISP does allow SMTP (which many do not) then your emails may still be blocked as "spam.").

Only way around all of this is to pay for a "business" internet connection (with 500% markup for the same speed, single dedicated IP, and a pretty poor SLA).

I pay about $20-25 more for my business line than residential service, for the same speed and 5 dedicated IPs. Yes, it's more, but it's not 500% more....

It really depends on the country you live in (and the provider you subscribe to).

As a bonus, you can actually get 24/7 support!

Unless you have Charter. Charter business support is open 9-5 facepalm

EDIT: (this was a few years ago, don't have charter business at the moment)

>with 500% markup for the same speed, single dedicated IP, and a pretty poor SLA).

This is overly cynical. There's a world of difference between your crappy Comcast line at home and my fiber line at work. Unless you're actually responsible for an infrastructure, then I don't think its easy to appreciate the difference, but yes, its worth every penny. You may not realize it but jitter, latency, routing, etc are far, far better on proper business lines. Not to mention SLA's, dedicated bandwidth, and actually having someone to yell at on the phone who can do something for you instead of being yet another anonymous schmuck on discounted service that's oversold and runs like crap.

Oh, at previous gigs we tried "business cable" which was laughable. All the downsides of Comcast but somehow even worse uptimes and incompetence.

Personally, I agree that smtp traffic originating from residential blocks should be weighed much heavier than other blocks. Wanna see my mail logs at my spam filter edge? Most Windows computer in the world has been taken over in some fashion by malware. Its worse than you think out there.

> Wanna see my mail logs at my spam filter edge?

As someone who is curious about running a mail server but worried about security this would be very interesting to have a grep at!

Pastebin?

I can't give real logs because of privacy concerns but in the past 7 days we've been at 79.2% spam. Roughly 2% of that is straight up malware like cryptolocker, various botnets, trojans, etc.

Shame no worries though.

Do you use spam assassin or something similar?

Comcast busses class still has all their IP's in many RBL's, this product won't work reliably for deliverability.

I've run a personal mail server on Comcast business class for about 2 years and haven't noticed this problem. Probably the majority of my outbound mail is to Gmail. I don't remember if I've checked the RBLs before, but I probably should re-check.

My place of business is on Comcast business and we score 10/10 on https://www.mail-tester.com/.

I have problems with several places that firewall the Comcast IPs - Symantec (messagelabs) is the worst because I don't know who uses their services until the message queue backs up and then I have to route around the damage. Really annoying.

To make it worse, I cannot find a way to contact them and ask them politely to whitelist our email server IP address.

Most mail services give a a rejection and, when I've asked to be whitelisted, they do it with no problem (Microsoft, for instance). Symantec simply doesn't respond on port 25 and has no contact information.

It isn't just me: https://www.google.com/search?q=messagelabs+port+blocked

The big providers are actually less of a problem when it comes to RBLs than smaller ones, as the big providers have their own filtering and/or rely on reputable RBLs. The bigger problem in my experience is companies with overzealous IT people with the authority to largely do as they please, where you get some percentage that pick their favorite set of block lists with no consideration for whether or not said block lists are properly run.

We occasionally get on block lists, and though we do get onto reputable block lists now and again after mistakes by users, they're rarely a problem and quick to get off. But now and again we get on some shady block list where there's no quality assurance, and where delisting procedures are onerous. Every now and again one of them is an outright money-making scam ("pay $99 for express delisting, or wait for X days for the regular service; oh by the way we just randomly added you with no basis")

You certainly should check. As I'm sure you are aware, some RBL's areole aggressive than others. Some are down-right draconian, and some are more like a whitelist in that they leverage grey listing so any outbound mail is given a few good points in it's total possible spam score.

I ran a medium email ( late 90's ) mail server system on a dedicated 100mb/s line, on an older dual cpu Power Mac actually, but I pulled out all the GUI and tried to run it mostly as a server using the client OS and only the CLI, making it more like running FreeBSD or openBSD I suspect.

I got a /24 IP range out of Comcast which I think helped as it was a previously never used /24, or at least not in use long enough that it had fallen out of RBL's

I'm sure you are aware of the sites that checks your IP against all 200+ or so mainstream RBL's.

There are a few RBL's that block Comcast business as well as all cellular ranges and quite a bit more. I found RBL's to be more trouble than worth.

Ideally, you are dropping the connection at the very beginning, just after the EHLO/HELO., highly efficient, perhaps marginally more efficient CPU-wise than greylisting. Fail2Ban is probably your best friend. Do it closer to the network if possible.

The above, versus pushing the message through various spam filtering, address validation, and other measures. There's heavy CPU usage post greylisting, greylisting is great, but waiting that 5-15 minutes for the retry is too long, most users don't use a password manager. Their password manager is "forgot your password". Insane, but that's how I watch all my friends do it. They learned not to re-use passwords, but now they rely on third party email to send forgot password messages. At least people can tell which are not hashing/+salting their credentials. Though I doubt most blink an eye at a raw password in an email. Clients have email me their credit card data. I tell them delete their debt message, then dived 15 minutes doing tech support to help them locate the credit card sent email.

I was far too often adding whitelist entries, making certain clients ignore greylisting or drop down to 30 seconds, for which I saw no increase in spam. It's the retry value on the server that's set too high/long. Not to mention the 10% or so of older email servers that don't support greylisting so never retry. A clear RFC violation, but these are proprietary systems and not made with greylisting in mind. You could achieve it with a proxy running greylisting or ASSP if you want a full proxy with web admin and essentially a full MTA in a proxy all in one file of many tens of thousands of perl code. No version control, crazy version numbers, and I think still on SFNet ( Source Forge )

At some point you realize you have added aol, yahoo, gmail, etc., all the big guys, they need whitelisting because their IP ranges are huge, and ever changing. Keeping on top of which IP's they are publishing as public MTA based IP's, you can build a solid whitelistable IP range, but they change often enough it's not feasible. A SAAS that checked all this would be nice but latency may be too much. Less than the 5-30 minutes of greylisting though.

AOL fully ignores greylisting last I looked. Not to mention DNS TTL's are/were ignored. Sucked to set my MX TTL to 300 seconds, wait the 8 hours for my default DNS expiry, switch my DNS, and then wait 2 weeks for DNS proposition with aol. Not to mention all the paperwork that needs filling out to get into their MTA whitelist program which kicks your sending threshold up by a percentage so it's not a true whitelist. I had to renegotiate every time we added another 5000 to our outbound, it would flag me and I could see all the aol messages stuck in my outgoing queue. Two weeks was longer than my greylisting time, messages bounced because aol saw no server at the other end. I had to keep a "gateway" running, basically I set up a secondary MX to sit there and deal with the aol 2 week DNS propagation to happen.

Finally I gave up and added an external SMTP only machine in a colo. small machine, under 100.00 a month, all it did was SMTP. Deliverability was significantly better after that but ruined the notion of securing my data at my location. But it is relatively ephemeral in that its just SMTP, the data sends instantly, unless it hits greylisting servers or other things that initiate a SMTP retry. That leaves messages and retry logs on the remote server out of my control physically.

Most don't answer postmaster@ or abuse@ which are RFC. Some even bounce meaning they don't even have those addresses available. Having a support/postmaster/abuse @ address that you certainly should check. As I'm sure you are aware, some RBL's are more aggressive than others. Some are down-right draconian, and some are more like a whitelist in that they leverage grey listing so any outbound mail is given a few good points in it's total possible span score.

I ran a medium email server system on a dedicated 100mb/s line, on an older Power Mac actually, but I Pulled out all the GUI and tried to run it mostly as a server using client OS and only the CLI, making it more like running FreeBSD or openBSD I suspect. Added 2x SATA cards and an additional CPU to speed things up. IMAP is purely I/O limited and I wanted and had everyone in IMAP. but IMAP needs a few horses behind it, I wish SSD's were where they are now back in the 90's. That would have been great. I moved my 4GB IMAP account into pure ram disc. It was pretty amazing. Millisecond loading of a mailbox with push enabled in milliseconds to load many tens of thousands of messages and attachments.

I got a /24 out of Comcast which I think helped as it was a previously never used /24, or at least not in use long enough that it had fallen out of RBL's

I'm sure you are aware of the sites that check your IP against all 200+ or so mainstream RBL's.

There are a few RBL's that block Comcast business as well as all cellular ranges and quite a bit more. I found RBL's to be more trouble than they are worth. Good in theory and a great thing to run your own local RBL as it's such a nice way to manage it, through DNS. Want to whitelist or block, just add a DNS entry after reversing the IP. Simple, immediate, understandable.

Ideally, you are dropping the connection at the very beginning just after the EHLO/HELO. Versus pushing the message through various spam filtering, address validation, and other measures. There's heavy CPU usage post greylisting, greylisting is great, but wIting that 5-15 minutes for the retry is too long, most users don't use a password manager. Their password manager is the "I forgot my password" mechanism.

I was far too often adding whitelist entries, making certain clients ignore greylisting or drop down to 30 seconds, for which I saw no increase in spam. It's the return value on the severe that's set too high/long.

Eventually I ran out of granularity per user to set things how I wanted. Setting up a server for a subset of users was asking for trouble.

At some point you realize you have added aol, yahoo, gmail, etc., all the big guys, they need whitelisting because their IP's are huge, and ever changing. Keeping on top of which IP's they are publishing as public MTA based IP's, you hBe a solid whitelist able IP, but they change often.

AOL fully ignores greylisting last I looked. Most don't answer postmaster@ or abuse@ which are RFC required for postmaster and suggested strongly for abuse@.

Oracle ended up in spamhaus, after weeks, I finally got I touch with their admin. ( grind worked there, could not email him ) He'd ( Oracle MTA Admin ) been trying to figure it out for 4 months of bounced emails. How they never had a forwarded email from a bounce that shows the 5.5.0 code is all they would have needed.

But that poses another problem. You have to whitelist abuse@ and postmaster@ or the data you send them will trigger everything, anti-spam, and bounce. But then you get tons of spam, fully unfiltered on those two accounts.

Then there's DKIM and the rest plus the rest of the DNS based pseudo AUTH mechanisms. But those too are problematic. Some can't forward without bouncing, under SPF I believe. Oracle ended up in spamhaus, after weeks, I finally got I touch with their admin. He'd been trying to figure it out for 4 months or bounced emails. How they never had a forwarded email from a bounce that shows the 5.5.0 code is all they would have needed.

But that poses another problem. You have to whitelist abuse@ and postmaster@ or the data you send them will trigger everything from anti-spam and bounce. But then you get tons of spam, fully unfiltered on those two accounts.

In the end, I'm basically tailing logs, massaging them into a SNMP walkable data point so I can graph and chart. All to find out managing a mail server really is a near full time job. Add a secondary and it's even worse as spammers will direct target your mx2 Which means filtering, whitelist, blacklist, greylisting, geographically separated, and network role separated ideally, etc., full parity on two physically disparate boxes.

I was pushing about a million messages a day across a few 10's of thousands of accounts, forwards, aliases, etc. all over Comcast business.

The upsides: 4 hour support window 24/7/365. Direct phone line that a person answers. That person is an engineer of some sort or will transfer you to someone. I've spoken to engineers that maintain DNS, mail, web, etc. you get as close to he source as possible and they know how to unpack a gzipped or tar'd log file batch if you can even get a contact address for them.

Sorry if there's duplication of content. Either something is up with HN or my phone is being my phone as usual. I apologize for the illegibility in places.

"Many ISP's" don't actually block 25/TCP outbound but simply add their residential netblocks to, e.g., the Spamhaus PBL. An end-user assigned one of these IPs can, upon request, have it removed from the PBL himself. This is, IMO, a much better way of "disallowing" outbound SMTP than simply filtering out packets.

I'm curious as to why this is better in your opinion. Doesn't adding the range to a blocklist carry the potential of being copied by other blocklists? Then an ip seeking to send mail must track down and request removal from every blocklist that it's been added to whereas the alternate approach requires the email administrator only contact a single entity, and one they are a customer of to boot.

If, as an end user, your IP address is on the Spamhaus PBL, you can have it removed yourself simply by submitting a request online.

If packet filters are used to block 25/TCP, you have to request that your ISP unblocks it, hope that they do, and then wait for them to actually do it.

I'm not sure its really a "server" in the way you're thinking. It looks more like a USB storage device to me.

> Ports 25, 443, 993 are needed in order to have a fully functional Own-Mailbox.

Even if you flout your ISP's server policy, there's still DHCP to contend with (changing IPs).

dynamic DNS exists. There's no reason why the host behind a MX entry can't be a dynamically updated address. MX records are names, not addresses.

Reverse DNS is a pretty big issue. You're not going to actually deliver any email without valid reverse DNS.

What is "valid"? I've been delivering personal email for 2 years from an IP whose reverse is like 1-2-3-4-static.hfc.comcastbusiness.net, and that name does not match what I put in my MX or HELO.

Now, granted, there are probably some systems that have issues with this, and maybe I do have failed deliveries I didn't notice, but you did say no email will deliver at all.

The mail systems that I run (at/for an ISP) would block your mail (although I am probably more strict about what I will accept than most).

Valid is generally the one that matches your HELO, and in a perfect world your machine's hostname.

I was under the impression that you just had to have an IP with an A record that matched the reverse DNS, even if it's not for the A record that's your email domain?

It depends on the MX you're delivering mail to. Some aren't restrictive at all, some will reject if there's an A/PTR mismatch, etc. I make use of FCrDNS on my mail systems.

From my initial read, seems like SMTP relay is the way to go with a device like this

It's a full, if small, computer, running Linux on an iMX233. Roughly equivalent to this sort of thing. https://www.olimex.com/Products/OLinuXino/iMX233/iMX233-OLin...

(I had to dig around in the BOM to find out that detail..)

In this case, this isn't necessarily needed as own-mailbox could be the MX and then transmit the data using some other protocol or even just SMTP over a different port.

Of course this means that they could read your email again, so we'd be back to square one.

Well, they could only read the HTTPS link. If they were to open it, you'd have at least a record of that.

But they tend to allow you to send and receive SMTP though their servers so for a thing like this there is only a potential problem with the web and IMAP part ... which can be solved with a VPN.

So your ISP is not providing you an internet access ? That's lame.

It's usually a market segmentation strategy. You have to pay for a "business" connection if you want to run a server.

Also, blocking standard email ports on residential connections is a common anti-spam measure.

It's an easy anti-spam measure for sure, but there's no reason you (as the user) shouldn't have a checkbox allowing you to remove this limitation. Otherwise it's not a full internet access.

There's a good reason: The demand for it is infinitesimally tiny, and the hassle of doing so (and having to deal with monitoring whether or not you end up being a spam source) is an extra cost.

An argument that it's not "full" internet access will get you nowhere - very few consumer internet subscriptions are "full" internet access in the sense that they are totally unrestricted - most have very severe restrictions on e.g. running servers and many other things.

They mean if you send mail to anyone else - their spam filter will block you for having a residential IP.

Email is a bit less useful when everyone has you auto-blocked for being 'spam'.

I'd forgotten about that part. Yes, having an IP from a pool marked as "dynamic" is enough to get put on many spam blacklists.

I'm not sure I understood you correctly. Are you saying it's a common thing for all "ISPs" in the US to block outgoing traffic on port 25 (and allow incoming traffic for Outlook et al.)? Or do you simply mean that most email providers will block incoming mails from residential IP adresses?

The first. I don't know exactly how many ISPs do it, but it's pretty common. http://www.vtdesignworks.com/blog/comcast-xfinity-port-25-bl...

My provider (BellSouth or AT&T or whatever they're calling themselves this week) block incoming port 25 and block outgoing to port 25 to anywhere except their SMTP servers, because of spam.

I'm fine with that, but then it is not an ISP.

Words mean what people use them for. Sure it's technically only 65534/65535 of an ISP, but that's not a very useful way to communicate with people about the issue.

Not allowing to run servers from home is especially vague. So it's not like if only port 25 is blocked

Are you allowed to run a game server, ftp server, web server, ssh server ?

Words first mean what is written in a dictionary or encyclopedia.

Internet : https://en.wikipedia.org/wiki/Internet

If you are not allowed to run server, they are WSP (Web Service Provider) not ISP.

But if you are OK with ISP not providing internet, I'm sure you are fine with unlimited limited or throttled after x, 24/24 access between 9-5 and secured http page as long as there is a padlock favicon ...

> Words first mean what is written in a dictionary or encyclopedia.

Nope - this is an important difference between English and other languages. If what people say differs from what's in the dictionary, it's the dictionary that will change, not the people.

True, maybe they are using "reverse VPN"? Works for me.

> Many ISP's including my own in the U.S. don't allow running servers from home, especially SMTP.

Land of the free! Start your own ISP! ;-)

Yeah, in my particular "land of the free" (the one everybody thinks of when you say that), there are actually laws on the books in many states and local jurisdictions which make it illegal to start your own cooperative/municipal ISP. I know the brochure says nice things about this country, but it's really not as great as it sounds.