News for hackers. Stuff that matters.
Re:Comment spam random text template by tikhonj on Monday April 22, 2013 @02:32PM (#5590963)
Sure, there can be millions of unique texts, but they all follow certain very similar patterns. That's what would make even a simple naive Bayes filter very effective against them.
Re:Comment spam random text template by klodolph on Monday April 22, 2013 @11:42PM (#5593175)
I think you're misunderstanding how naïve Bayesian spam filters work. These techniques explicitly do not recognize patterns at all, they just look at vocabulary, and treat each word as a completely independent feature—so they don't even recognize "discount viagra" as a pattern, but merely two independent features "discount" and "viagra".
Re:Comment spam random text template by tikhonj on Tuesday April 23, 2013 @03:13AM (#5593728)
First of all, you can have any sort of features you want. So you could certainly look at things like sentence structure, pairs of words and so on.

More importantly, I used "pattern" in a very general sense: there is strictly more structure to a post generated with this system than a random post or even a human-authored one. So repeated words between posts are a pattern, and a very important one at that.

Re:Comment spam random text template by robrenaud on Tuesday April 23, 2013 @02:25AM (#5593575)
You can build a naive bayes classifier with n-gram features.