News for hackers. Stuff that matters.
Re:Messing with Scraper Bots by mnau on Saturday November 15, 2025 @07:35PM (#45941755)
He addresses that. Basically, there are gatekeepers and if you get on the wrong side of them, only manual intervention can save you. And we all know how Google loves providing a human to resolve problems.

> I came to the conclusion that running this can be risky for your website. The main risk is that despite correctly using robots.txt, nofollow, and noindex rules, there's still a chance that Googlebot or other search engines scrapers will scrape the wrong endpoint and determine you're spamming.

There are no comments on this comment yet.