News for hackers. Stuff that matters.
Re:Avoiding Webscraping Throttling Using Python and Tor as a... by nullwasamistake on Sunday July 14, 2019 @04:00AM (#20432309)
Your site is public, what do you expect? It's not abuse to load public websites. T&C is meaningless for public sites. Equivalent to real life sign hanging in a shop "whoever reads this is bound to the following conditions". That's not how contracts work

If you block the small guys from scraping you're just helping already huge monopolies tighten their grip on the content market. Even "nice" scrapers like Google blatantly steal your content, blocking everyone else just makes their monopoly worse. They steal pricing data for "Google Shopping" regularly.

If it annoys you that much just build an API and charge tiny amount to use it.

There are no comments on this comment yet.