Re:Tell HN: We Need Robots.txt but for AI
by donohoe on Tuesday January 10, 2023 @08:59AM
(#34324896)
You can do this but it has no power.
You can forbid and dictate terms but you are unable to enforce it, just like those email footers are a waste of space and have no enforcement power whatsoever.
robots.txt only works because the search engines agree to follow it. Likewise, the DNT browser setting is ignore by most despite it being a loud and clear signal to user intent.
You can forbid and dictate terms but you are unable to enforce it, just like those email footers are a waste of space and have no enforcement power whatsoever.
robots.txt only works because the search engines agree to follow it. Likewise, the DNT browser setting is ignore by most despite it being a loud and clear signal to user intent.