Re:We should start to add “ai.txt” as we do for “robots.txt”
by alwayslikethis on Wednesday May 10, 2023 @09:08AM
(#35886798)
Why not serve fake garbage indistinguishable from real content by a computer, like LLM output? Sending errors just incentivizes bot owners to fix the identifiable parts
[ Parent
| on HN ]
Re:We should start to add “ai.txt” as we do for “robots.txt”
by ape4 on Wednesday May 10, 2023 @09:41AM
(#35887222)
I like this idea. Of course it would have to be only to robots that visit a page disallowed by the robots.txt
Re:We should start to add “ai.txt” as we do for “robots.txt”
by shanebellone on Wednesday May 10, 2023 @09:28AM
(#35887048)
"Why not serve fake garbage indistinguishable from real content by a computer, like LLM output?"
Serving more than the minimum wastes resources. Worse yet, a better solution would cost my time.
"Sending errors just incentivizes bot owners to fix the identifiable parts"
Sure, someone could make or configure their scraper perfectly. "Perfect" is now the table stakes though.
Edit:
My solution strives to cause an unproportional expense in order to circumvent. I want 10x on my time.
Re:We should start to add “ai.txt” as we do for “robots.txt”
by twelve40 on Wednesday May 10, 2023 @09:31AM
(#35887091)
it'd be cool to be able to fingerprint that garbage, too. Like, sprinkle some hashes here and there (or something like that) so that you can later uniquely look up your own "content" being stolen by chatbots and which ones.
Re:We should start to add “ai.txt” as we do for “robots.txt”
by shanebellone on Wednesday May 10, 2023 @09:36AM
(#35887160)
You can. I can't think of the appropriate term though. Hopefully someone else chimes in here with a link.