Re:A Look Inside Our 210TB 2012 Web Corpus
by greglindahl on Tuesday August 13, 2013 @11:25PM
(#6209784)
blekko attempted to implement crawling a local copy, and it was a PITA. We'd rather crawl the real thing with a crawl-delay of 1. Best would be if the Wikimedia Foundation made a .html dump available.
[ Parent
| on HN ]
There are no comments on this comment yet.