News for hackers. Stuff that matters.
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by yarapavan on Wednesday August 31, 2016 @07:54AM (#12397578)
Tarantool, mentioned in here, is an in-memory database and application server with 100% compatible drop-in replacement for Lua 5.1

Github page: https://github.com/tarantool/tarantool/

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by kazinator on Wednesday August 31, 2016 @11:18AM (#12398943)
One powerful rule that I use in my Exim setup is to drop any SMTP connection that doesn't have forward and backward DNS that match. This catches a big majority of all spam, with hardly any false positive downside. No credible in SMTP forwarding host has broken DNS (by definition, practically). Lots of spamming machines do. Of course, not all, but this is a game of percentages and multiple stages.
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by treve on Wednesday August 31, 2016 @11:45AM (#12399145)
You'll definitely lose email from Amazon SES then. What about checking SPF records instead?
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by kazinator on Wednesday August 31, 2016 @11:55AM (#12399230)
My darned residential cable line has forward and reverse DNS that match.

If the Amazon SES can't figure this out, they have no business sending SMTP e-mail.

I do apply the SPF check and reject the SMTP if it comes up as a confirmed negative: the sending domain has an SPF record, with strict semantics, and the IP doesn't match.

SPF speaks to a different problem (not the problem of "is this host a spammy host?"), and generally has no teeth. Passing SPF only informs me that the IP address sending to me is allowed to use the given envelope sender domain. A spammer can register a bunch of domains and set up SPF records for them, and then use the domains as envelope addresses. (Of course, these envelope sender addresses don't necessarily have anything to do with what is in the From: header).

You can't expect some sort of free pass just because you made a SPF record. What the SPF record does is protect your domain from being used by other people the bases of a sender envelope address. This is of low value, because most recipients don't even see the envelope address.

SPF can be applied against From: too but that's fraught with problems, because of forwarding via mailing lists, or people using alternative sending identities which are legitimate.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by r00fus on Wednesday August 31, 2016 @06:51PM (#12402072)
Is this a bad thing, if you're trying to avoid spam?
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by marcosdumay on Wednesday August 31, 2016 @04:35PM (#12401261)
Never thought about it (and I've been thinking about email a lot recently), but now that people mentioned it...

Every mail sever defaults should be to trust an encrypted connection with a valid MX host name. Failing that, it may choose to check either reverse DNS or SPF. I see no reason for preferring any of those. Not checking anything by default isn't a good idea nowadays.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by Nux on Wednesday August 31, 2016 @12:00PM (#12399274)
So, do you require the PTR and A records of the remote host to match?
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by kazinator on Wednesday August 31, 2016 @12:48PM (#12399659)
I don't work with DNS at the protocol level; it's done via the Exim "verify = reverse_host_lookup". The documentation says:

This condition ensures that a verified host name has been looked up from the IP address of the client host. (This may have happened already if the host name was needed for checking a host list, or if the host matched host_lookup.) Verification ensures that the host name obtained from a reverse DNS lookup, or one of its aliases, does, when it is itself looked up in the DNS, yield the original IP address.

The phrase "or one of its aliases" seems to imply that the match does not strictly require the A record.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by jlgaddis on Wednesday August 31, 2016 @09:44PM (#12402683)
The general term for this is "Forward Confirmed reverse DNS" (FCrDNS). I do these checks (plus others) on my Postfix mail systems. I would assume most of the other (sane) MTAs can perform these checks as well.
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by kazinator on Wednesday August 31, 2016 @11:26AM (#12398997)
I use a home-grown web app called Tamarind for managing throwaway mail aliases, in conjunction with Exim:

http://www.kylheku.com/cgit/tamarind/tree/README

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by Robin_Message on Wednesday August 31, 2016 @12:50PM (#12399683)
The spam bit sounds cool but the nosql db sounds odd. E.G. That tree implementation with batched rebalancing sounds downright scary, and I thought avl was considered the worst kind of balanced tree due to excessive overhead (an int per node to red-black's bit) .
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by bigbes on Wednesday August 31, 2016 @06:57PM (#12402094)
> Our investigation revealed that in case of frequent insertions and deletions Tarantool initiated a complex process of tree rebalancing (all our indexes were of TREE type).

It's all about 1.5. New version (1.6) uses brand new bps-tree, not sg/avl-tree. It behaves better on all workloads.

AVLTree was "temporary" hack. Our implementation works better, for their needs. BTW - AVL is not bad, but it's hard to implement a good one (believe me :) ).

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by Robin_Message on Thursday September 01, 2016 @05:30AM (#12403938)
Nice. Yeah, AVL is definitely okay but it makes me happy to hear something better is in there :)

What sort of tree is this new bps-tree?

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by bigbes on Thursday September 01, 2016 @05:33PM (#12409074)
bps-tree means B+-tree: unique combination of B+ and B tree. You can read wiki or "The Ubiquitous B-Tree" whitepaper https://wwwold.cs.umd.edu/class/fall2002/cmsc818s/Readings/b... .

You may read code of it here: https://github.com/tarantool/tarantool/blob/1.6/src/lib/sala... . It's very thoroughly commented.

[dead]
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by dang on Wednesday August 31, 2016 @12:29PM (#12399510)
Personal attacks are not allowed on Hacker News. We ban accounts that do this, so please don't do it again, even at someone you feel is unskilled at setting up a mail server.

We detached this subthread from https://news.ycombinator.com/item?id=12398943 and marked it off-topic.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by ihuman on Wednesday August 31, 2016 @01:42PM (#12400076)
I understand why this comment was deleted, but how come it was detached and made into a top-level thread? I'm not against it or anything (if I sound like I am), I just don't see anything in the rules or FAQ about it, so I was curious.
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by dang on Wednesday August 31, 2016 @01:53PM (#12400149)
We do that because then we can mark the subthread off-topic, putting it much lower on the page. If we didn't detach it first, marking off-topic would only put it lower among its siblings.

This originated as a technique for dealing with "topjacking", i.e. comments that are posted as a reply to the top comment, not because they're relevant replies but simply to be nearer the top of the page. Later, we realized it was useful to demote subthreads where people were breaking the HN guidelines.

When we do this, we nearly always post a comment explaining that we did it. For mind-numbingly boring reading, see https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=... and https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=....

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by Normal_gaussian on Wednesday August 31, 2016 @03:11PM (#12400686)
Those may only be mind numbing for you who have to go through it every day, I really enjoyed the first two pages. Couldn't imagine having to do the replies though - a suprisingly large number had been customised!
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by dang on Wednesday August 31, 2016 @03:46PM (#12400939)
> I really enjoyed the first two pages

Ha! Well that made me laugh out loud.

Yes, we write all those comments by hand and often make a point of customizing them. I even plant little bits of entertainment in there sometimes (hopefully not at anyone's expense), on the theory that the hivemind enjoys tidbits of variety.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by dmm on Wednesday August 31, 2016 @03:49PM (#12400955)
Sorry dang, kazinator.
Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by kazinator on Wednesday August 31, 2016 @11:57AM (#12399243)
Resolving MX records from your home and contacting hosts directly on their port 25 is a nonstarter. I wouldn't even think of doing it. Instead, I forward to my ISP's SMTP forwarding hosts (via an authenticated SMTP connection).

It's not necessarily the "working DNS" rule that will reject you, but other methods, like anti-spam databases that list residential line IP addresses. For example, my residential line, in fact, has forward and backward DNS that match. If I directly contact someone's mail server which has this rule, I can get past that rule. That's not what will "get" me.

Re:The Cat-And-Mouse Story of Implementing Anti-Spam for Mai... by AstralStorm on Wednesday August 31, 2016 @12:44PM (#12399640)
Funny thing, my own old mail servers allowed that for AUTHed SMTP w/ correct TLS, but then disallowed invalid domains or wrong account in From field.

There are really no legitimate reasons to allow spoofed outgoing mail.