News for hackers. Stuff that matters.
Re:The Cost of Being Crawled: LLM Bots and Vercel Image API ... by randunel on Tuesday April 15, 2025 @03:54AM (#43690118)
Replace "podcasts" with "search results" in your comment, and "RSS feed" with "LLM output" and you've got yourself the exact same argument for what's going on today. The company names are different, of course, but not by much because some of the players stayed the same.

Your lack of reply to "do you observe robots.txt when you download content such as images" is basically a "no".

There are no comments on this comment yet.