gigaArpit
The Mozilla VPN Extension for Windows
Google does not want rights to things you do using Chrome (2008)
OpenAI Roadmap Update for GPT-4.5 and GPT-5
Personal Software: The Unbundling of the Programmer
Re:AI bots everywhere. Does anyone have a good whitelist for robots.txt?
on Wednesday January 29, 2025 @12:01AM (#42861738)
Robots.txt is a joke, use Cloudflare's Block AI Bots feature if you are using Cloudflare.
Re:We Need a New HTTP Code: "Unavailable to AI"
on Sunday January 26, 2025 @03:54PM (#42833991)
Following this reasoning, a 200 response would also be appropriate, as AI bots wouldn't flag anything as suspicious.
Re:We Need a New HTTP Code: "Unavailable to AI"
on Sunday January 26, 2025 @03:38PM (#42833835)
The idea behind introducing a new HTTP status code for blocking AI traffic should be to provide a clearer signal to both users and automated systems about the nature of the response. While a 404 indicates that a resource is not found, it doesn't convey the specific reason that the content is being restricted from AI access.
While User-Agents can easily be spoofed by users and automated traffic, as you mentioned in your earlier reply, the HTTP status code would be a clear message from the server.
Re:We Need a New HTTP Code: "Unavailable to AI"
on Sunday January 26, 2025 @03:26PM (#42833727)
Each code has a defined meaning, which helps clients and servers understand the nature of the response.