News for hackers. Stuff that matters.

A comment on We should start to add “ai.txt” as we do for “robots.txt”. Parent comment

Re:We should start to add “ai.txt” as we do for “robots.txt” by none_to_remain on Wednesday May 10, 2023 @10:05AM (#35887550)
Robots.txt is meant as an aid to crawlers. "This stuff is not useful to index," rather than a blocking mechanism
[ Parent | on HN ]

There are no comments on this comment yet.