News for hackers. Stuff that matters.

A comment on The State of Web Scraping 2022. Parent comment

Re:The State of Web Scraping 2022 by rronalddas on Thursday January 13, 2022 @11:30AM (#29922647)
They read robots.txt right? You can easily add a Disallow rule for google-bot there
[ Parent | on HN ]

There are no comments on this comment yet.