Re:Cloudflare Introduces Default Blocking of A.I. Data Scrapers
by Dylan16807 on Thursday July 03, 2025 @05:19PM
(#44459288)
That is not when they started recommending it. It would be nice if you acknowledged the rest of my comment, I even quoted from the [8] reference.
On July 1, 2019, Google announced the proposal of the Robots Exclusion Protocol as an official standard under Internet Engineering Task Force.[8]
https://en.wikipedia.org/wiki/Robots.txt