News for hackers. Stuff that matters.
Re:Google competitor DuckDuckGo says it's getting shut out by Matt_Cutts on Thursday November 22, 2012 @11:37AM (#4819267)
That's interesting.

Gabriel, does DuckDuckGo's crawler have a distinct user agent? Can you talk more about how DuckDuckGo observes/respects robots.txt?

There are no comments on this comment yet.