Re:Show HN: CloudScrape – Cloud-based web scraping platform
by snehesht on Saturday October 17, 2015 @12:47PM
(#10405012)
No offense but ...
First thing, it's just a suggestion. Don't use duplicate account to reply to the comments. Do it from your original account.
I think what the user "chdir" means by wrong is that you're not honoring "robots.txt" and how do you account for ethical scraping ( eg. running 50 concurrent connection on a single website and overloading the website, technically its DoS attack)
First thing, it's just a suggestion. Don't use duplicate account to reply to the comments. Do it from your original account.
I think what the user "chdir" means by wrong is that you're not honoring "robots.txt" and how do you account for ethical scraping ( eg. running 50 concurrent connection on a single website and overloading the website, technically its DoS attack)