News for hackers. Stuff that matters.
Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by sbarre on Tuesday April 08, 2025 @08:38AM (#43621020)
LLMs are powered by web scraping.

The same way Google and others have been crawling and capturing all your public posts for decades to power their search engines. Now the data is being used to power LLMs.

Were you able to opt out of being part of the search index (and I don't mean at the site level with a robots.txt file)?

I think your choice here is "don't post on a publicly accessible website", unfortunately.

Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by diggan on Tuesday April 08, 2025 @09:03AM (#43621307)
> Were you able to opt out of being part of the search index (and I don't mean at the site level with a robots.txt file)?

If you're in the EU, then yes, as "Right to be Forgotten" is a thing: https://en.wikipedia.org/wiki/Right_to_be_forgotten#European...

But in general I agree, the expectation of something remaining "private" and "owned by you" after you publish it on the public internet, should be just about zero. Don't publish stuff you don't want others to read/store/redistribute/archive.

Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by voidUpdate on Tuesday April 08, 2025 @09:18AM (#43621471)
I manually opted in to being on the search index by submitting my website to google. I have never opted in to being part of an LLM dataset
Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by sbarre on Tuesday April 08, 2025 @11:49AM (#43623165)
Maybe I misunderstood your original post, I thought you meant your comments here on HN, not a personal website you control.

Others have said it already but when you are posting here on a public website, I would argue that you are effectively consenting that your content is now available for consumption by site visitors.

"Site visitors" may include people, systems, software, etc..

I think it would be pretty impractical for every visitor to the site to have to seek consent from each poster before making use of the content. That would literally break the Internet.

Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by Daviey on Tuesday April 08, 2025 @09:41AM (#43621669)
Which search engines are you in that you didn't opt into?
Re:LLM-hacker-news: LLM plugin for pulling content from Hack... by voidUpdate on Tuesday April 08, 2025 @09:51AM (#43621793)
My website currently shows up on bing (which has an opt-out tool, I just haven't bothered to use it), duckduckgo (which scrapes its results from other engines) and yahoo search (which apparently scrapes from bing). I can't check Kagi as I'm not paying for it, and I can't currently think of other search engines off the top of my head