Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on AI bots everywhere. Does anyone have a good whitelist for robots.txt?. Parent comment
But, bot fight mode says "there is a newer version of this setting" however it does not link to it.
Anyone have any insight on the blocked verified bots or the supposed new version?
There are no comments on this comment yet.
But, bot fight mode says "there is a newer version of this setting" however it does not link to it.
Anyone have any insight on the blocked verified bots or the supposed new version?