News for hackers. Stuff that matters.
Re:I just finished crawling 5.19B web pages, Ask Me Anything by dor_jack_2 on Wednesday April 19, 2017 @07:40PM (#14152761)
For our purposes Common Crawl's corpus was missing too many websites (possibly due to robots.txt configs of websites) Also we needed some deep coverage which CC could not provide.

There are no comments on this comment yet.