News for hackers. Stuff that matters.

A comment on I just finished crawling 5.19B web pages, Ask Me Anything. Parent comment

Re:I just finished crawling 5.19B web pages, Ask Me Anything by savethefuture on Wednesday April 19, 2017 @07:13PM (#14152619)
How did you crawl so many sites, how did you discover them, search engine, ip ranges or another method?
[ Parent | on HN ]
Re:I just finished crawling 5.19B web pages, Ask Me Anything by dor_jack on Wednesday April 19, 2017 @08:54PM (#14153126)
The platform we used provided their own seedlist and took it from there.
[ Parent | on HN ]