Re:Google's updated privacy policy states it can use public ...
by JohnFen on Wednesday July 05, 2023 @01:29PM
(#36603761)
The problem with robots.txt is that it depends on the crawler to respect it (and many do not). It's not an adequate protection against crawlers in general and a specific form just for AI wouldn't be adequate either, for the same reason.
[ Parent
| on HN ]
There are no comments on this comment yet.