News for hackers. Stuff that matters.

A comment on Google's fact-checking bots build vast knowledge bank. Parent comment

Re:Google's fact-checking bots build vast knowledge bank by yutah on Friday August 22, 2014 @01:10PM (#8212950)
Which large website did you get banned from for scraping? I did some scrapping and never got banned... perhaps my scraping's rate was not too fast.
[ Parent | on HN ]
Re:Google's fact-checking bots build vast knowledge bank by adventured on Saturday August 23, 2014 @05:47AM (#8215409)
If you try to mass scrape almost any major site (millions of pages of content) they'll block you.

For example, if you went one by one through Stack Overflow and sucked out every question and answer, your scraper bot would get banned (unless you're doing one request per minute, in which case you'll never finish).

Or if you tried to scrape Twitter.

[ Parent | on HN ]