News for hackers. Stuff that matters.
Re:Blekko already has a list of domains to block from results by benbeltran on Friday January 07, 2011 @02:01AM (#2078875)
Two big problems with automated spam blocking are: false positives and changing domain names.

For the second one, how often do you revise your blocked links? what if it changed owner and the new one doesn't provide spam.

For the first one, is even one false positive tolerable? Will you deny someone presence in your index because you failed? And if so, how do you handle challenges?

Re:Blekko already has a list of domains to block from results by greglindahl on Friday January 07, 2011 @04:09PM (#2081129)
We don't mark a domain as spam until many of the pages we've seen look spammy.

Our ideal is to recrawl everything every 14 days, but during our launch we have not been achieving that.