News for hackers. Stuff that matters.
Re:The State of Web Scraping 2022 by RobSm on Thursday January 13, 2022 @02:09AM (#29917712)
So in the scale of google, 'not many' would be some few million per month? And all is good then, right? Even you use their scrapped data probably daily and are totally fine with that, right?

You think google bots read contracts before scraping website? really? :) If you had any experience in creating websites and launching them online, you would know how fast and often they arrive and how they do not care about your TOS. So the real 'violation' numbers might be very scary...for you.

https://ironcladapp.com/journal/contract-management/are-brow...

Re:The State of Web Scraping 2022 by rronalddas on Thursday January 13, 2022 @11:30AM (#29922647)
They read robots.txt right? You can easily add a Disallow rule for google-bot there