News for hackers. Stuff that matters.

A comment on We should start to add “ai.txt” as we do for “robots.txt”. Parent comment

Re:We should start to add “ai.txt” as we do for “robots.txt” by hiddencost on Wednesday May 10, 2023 @11:50AM (#35889105)
The point of robots.txt is to inform well behaved scrapers about how to behave. It is not designed nor intended to prevent bad actors.

Which is good design: don't pretend to solve problems you can't.

[ Parent | on HN ]

There are no comments on this comment yet.