News for hackers. Stuff that matters.
Re:How to Crawl the Web Politely with Scrapy by emodendroket on Thursday August 25, 2016 @01:49PM (#12360986)
It's certainly more considerate if you limit your scraping to rates similar to a human user instead of just going as fast as possible.

There are no comments on this comment yet.