Re:Ask HN: Get feedback on my bootstrapped web scraping as a...
by notastartup on Wednesday September 24, 2014 @10:15PM
(#8365209)
No, it will not fly through thousands of pages at a blinding rate using multiple ip addresses, it goes about at a human like pace to avoid getting banned. If an ip is banned a new one may be assigned. If you need to go faster you will need concurrent jobs.
All the jobs use chrome browser, and supports any website.
All the jobs use chrome browser, and supports any website.