Re:Blekko already has a list of domains to block from results
by greglindahl on Friday January 07, 2011 @04:09PM
(#2081129)
We don't mark a domain as spam until many of the pages we've seen look spammy.
Our ideal is to recrawl everything every 14 days, but during our launch we have not been achieving that.
For the second one, how often do you revise your blocked links? what if it changed owner and the new one doesn't provide spam.
For the first one, is even one false positive tolerable? Will you deny someone presence in your index because you failed? And if so, how do you handle challenges?