Re:How to crawl a quarter billion webpages in 40 hours
by wtf242 on Saturday August 11, 2012 @10:41AM
(#4370177)
here's how google avoids duplicate content: http://infolab.stanford.edu/~manku/papers/07www-duplicates.p...
[ Parent
| on HN ]
There are no comments on this comment yet.