News for hackers. Stuff that matters.
Re:Google's updated privacy policy states it can use public ... by JohnFen on Wednesday July 05, 2023 @01:29PM (#36603761)
The problem with robots.txt is that it depends on the crawler to respect it (and many do not). It's not an adequate protection against crawlers in general and a specific form just for AI wouldn't be adequate either, for the same reason.

There are no comments on this comment yet.