Re:Comment spam random text template
by lenazegher on Monday April 22, 2013 @02:29PM
(#5590944)
I don't know how easy it is to train a spam filter on, but nested spintax quickly scales to millions of unique output texts. It can be spun at the paragraph, sentence and word level. It's really nasty stuff.
[ Parent
| on HN ]
Sure, there can be millions of unique texts, but they all follow certain very similar patterns. That's what would make even a simple naive Bayes filter very effective against them.
I think you're misunderstanding how naïve Bayesian spam filters work. These techniques explicitly do not recognize patterns at all, they just look at vocabulary, and treat each word as a completely independent feature—so they don't even recognize "discount viagra" as a pattern, but merely two independent features "discount" and "viagra".
First of all, you can have any sort of features you want. So you could certainly look at things like sentence structure, pairs of words and so on.
More importantly, I used "pattern" in a very general sense: there is strictly more structure to a post generated with this system than a random post or even a human-authored one. So repeated words between posts are a pattern, and a very important one at that.