News for hackers. Stuff that matters.
Re:Germany's Schleswig-Holstein Completes Migration to Open ... by cyberax on Sunday October 12, 2025 @05:18PM (#45561995)
You can thank AI bots for that.
Re:Germany's Schleswig-Holstein Completes Migration to Open ... by zenmac on Sunday October 12, 2025 @08:15PM (#45563363)
Wow didn't realize it has comes to this. I guess the AI bots are ignoring the robots.txt now days huh, so just removing the index seems enough to stop the AI scrappers?
Re:Germany's Schleswig-Holstein Completes Migration to Open ... by cyberax on Monday October 13, 2025 @01:38AM (#45565006)
Yep. I have my own small gitweb server, and I was getting literally thousands requests per second from bots who try to crawl through all the links.