News for hackers. Stuff that matters.
Re:Scrapy Tips from the Pros by binarysolo on Wednesday January 20, 2016 @08:30PM (#10942775)
Seconding the practice of collect, then parse.

Storage is cheap relative to hammering a server (or working around the bounds of rate-limitation), so always save your collected things, then crawl it as many time as you desire as you play around with the right scrapes.

There are no comments on this comment yet.