News for hackers. Stuff that matters.

A comment on We should start to add “ai.txt” as we do for “robots.txt”. Parent comment

Re:We should start to add “ai.txt” as we do for “robots.txt” by jefftk on Wednesday May 10, 2023 @09:38AM (#35887183)
> It's not a system that we should copy for preventing the use of content in training AI

I don't see the OP saying anything about "ai.txt" being for that? They're advocating it as a way that AIs could use fewer tokens to understand what a site is about.

(Which I also don't think is a good idea, since we already have lots of ways of including structured metadata in pages, but the main problem is not that crawlers would ignore it.)

[ Parent | on HN ]
Re:We should start to add “ai.txt” as we do for “robots.txt” by kmoser on Wednesday May 10, 2023 @11:57AM (#35889221)
Not only do we already have lots of ways of including structured metadata, but if you want to include directives about what should/shouldn't be scraped and by whom, we already have robots.txt.

In other words, there's no need to create an ai.txt when the robots.txt standard can just be extended.

[ Parent | on HN ]