News for hackers. Stuff that matters.
Re:Cursor CLI by dotancohen on Friday August 08, 2025 @05:09AM (#44834996)
There's a reason we still call the file robots.txt by that name, and not web-scraping.txt or search-engines.txt.
Re:Cursor CLI by adobrawy on Friday August 08, 2025 @09:29AM (#44836704)
Yea, the reason is RFC9309.
Re:Cursor CLI by dotancohen on Friday August 08, 2025 @11:40AM (#44838329)
That's actually a good point, but we have RFCs superceding RFCs all the time. My favorite RFC, 8601, superseded I think half a dozen other RFCs.