News for hackers. Stuff that matters.
Re:How to crawl a quarter billion webpages in 40 hours by mrkrwtsn on Saturday August 11, 2012 @11:53PM (#4371763)
You could use an edit distance algorithm instead of using checksums. Although that would be really time intensive.

There are no comments on this comment yet.