News for hackers. Stuff that matters.
Re:Build a crawler to crawl million pages with only one mach... by j_s on Tuesday February 28, 2017 @01:42PM (#13756433)
Nice! (compared to the article's 100,000 pages / 15 minutes / 60 seconds = ~110 pages per second)

Did you happen to track memory usage at all? It would take a while to settle down, for sure.

I'm always interested in the amount of overhead Docker brings to the table. No biggie either way, thanks for sharing these details.

There are no comments on this comment yet.