News for hackers. Stuff that matters.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @11:30AM (#40031816)
Model collapse doomers are such an odd bunch. It's as if they think that the humans training the models have no agency over the data they use, or how that data is processed/weighted/etc.
Re:LLMs Are This Close to Destroying the Internet by nicklecompte on Sunday April 14, 2024 @12:20PM (#40032215)
Taken literally this comment suggests that OpenAI et al actually want racist 4chan rants in ChatGPT's training data, or illegal child pornography in DALL-E's training data. Obviously that's not the case: the problem is that it's impossible to effectively audit the amount of data required to make generative AI work.

If Big Tech can't even catch pornography or explicit uses of the n-word, there's no way they'll be able to filter subtle LLM / art generator hallucinations.

Re:LLMs Are This Close to Destroying the Internet by jsheard on Sunday April 14, 2024 @12:31PM (#40032309)
Going one level deeper, even AI-generated training data which doesn't contain hallucinations or other obvious errors could still negatively affect the performance of a new model. GenAI models have a tendency to repeat the same cliches, in sentence structure or the composition of images or whatever else, so even if their output is "fine" in the sense that it's factually correct and everyone has the right number of fingers, a model trained on it will still internalize those cliches and make them even more likely to show up again. That's even harder to audit out because it would only become apparent in aggregate when you realise your new model has learned to interpret "fantasy artwork" as "fantasy artwork in the generic Midjourney RLHF style" since it was trained on so many examples of that.
Re:LLMs Are This Close to Destroying the Internet by wruza on Sunday April 14, 2024 @12:51PM (#40032470)
That's really just a feature of the default system prompt, although ubiquitous. You can character-prompt local LLMs as you like, but no one seems to do that.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @12:54PM (#40032488)
ChatGPT doesn't take character prompts very well, that's why people don't know. Claude on the other hand is a champ with characters.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @12:43PM (#40032394)
Not so much that, as OpenAI has finite resources and is allocating them to areas that are more valuable.
Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @12:42PM (#40032383)
Has it been demonstrated that there is illegal CSAM in the DALL-E training?
Re:LLMs Are This Close to Destroying the Internet by jsheard on Sunday April 14, 2024 @12:49PM (#40032451)
I don't think it has, if only because OpenAI aren't transparent about where they get their image training data from, but the most widely used public image generation dataset was found to contain CSAM: https://www.theverge.com/2023/12/20/24009418/generative-ai-i...
Re:LLMs Are This Close to Destroying the Internet by nicklecompte on Sunday April 14, 2024 @01:24PM (#40032688)
It hasn't been demonstrated in DALL-E, but this was a problem in older OpenAI text-video models, and it's in plenty of CommonCrawl/etc data. There is anecdata from contractors suggesting that DALL-E generated CSAM imagery during RLHF, which is circumstantial evidence that similar images are somewhere in the training data. I think it's fair to say we can't rely on OpenAI's technical competence to remove this stuff from their training data, despite claims to the contrary w.r.t DALL-E 2. (Likewise it seems Copilot Designer churns out CSAM like candy.)

Your attitude is naive and competely at odds with (honest) AI safety research. The rational thing for users is to assume DALL-E 2 is contaminated by CSAM unless OpenAI can publicly demonstrate otherwise. Their leadership is too stupid and dishonest for people to take their word.

https://ieeexplore.ieee.org/abstract/document/9423393

https://arxiv.org/pdf/2110.01963.pdf?trk=public_post_comment...

https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...

Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @03:07PM (#40033519)
You're saying my attitude is "naive". I'm saying that it's irresponsible for you to state that there is CSAM in DALL-E's training data unless you can back that up.

I find OpenAI's refusal to document what's in the training data infuriating, but I that doesn't lead me to assume they weren't able to filter out CSAM from it.

If they don't care about this issue at all, why are they exposing low paid workers in Kenya to these conditions?

Re:LLMs Are This Close to Destroying the Internet by candiodari on Sunday April 14, 2024 @12:46PM (#40032422)
Why would you need to?

Obviously generalizing over pictures of humans and pictures of porn will enable CSAM generation ...

Plus a LOT of things have been declared illegal pornographic material in various places: all porn (China), drawn CSAM, any kind of drawn porn with unclear ages, ...

Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @01:01PM (#40032521)
> "Why would you need to?"

If someone is going to suggest that there is "illegal child pornography in DALL-E's training data" I think it needs to be backed up - that's a pretty big accusation.

Re:LLMs Are This Close to Destroying the Internet by andy99 on Sunday April 14, 2024 @11:39AM (#40031904)
I feel like the take there is that they're destroying the internet from the perspective of people who want to make money off of add revenue. Cynically, it's like they want the internet to be kind of bad - as in distraction from ads, begging for money and content arranged to support that, but not so bad it puts people off entirely or drives the incremental value of putting out "content" to zero because of automation.

For people writing on the internet for motivations other than ad-driven monetization, and those who read their stuff, there is basically no issue.

Re:LLMs Are This Close to Destroying the Internet by beej71 on Sunday April 14, 2024 @12:00PM (#40032067)
> For people writing on the internet for motivations other than ad-driven monetization, and those who read their stuff, there is basically no issue.

This is the key, I think. But it's small--The Small Web. I think we'll be better off, though.

Have a day job and produce quality content in your spare time, is my motto. A few billion people producing content in their spare time is still more than we can ever hope to consume.

Re:LLMs Are This Close to Destroying the Internet by grugagag on Sunday April 14, 2024 @11:45AM (#40031953)
> For people writing on the internet for motivations other than ad-driven monetization, and those who read their stuff, there is basically no issue.

I doubt these users will not experience a drop in quality as most authorship will roll out LLM laden content which will add more noise to the signal.

Re:LLMs Are This Close to Destroying the Internet by davidcbc on Sunday April 14, 2024 @12:25PM (#40032255)
I feel like it's the exact opposite, LLMs are great for ad revenue because you can create a ton of garbage and monetize it in a fraction of the time it takes actual content creators to make their content. Meanwhile, the actual good content is drowned in a sea of garbage and much harder to find
Re:LLMs Are This Close to Destroying the Internet by stavros on Sunday April 14, 2024 @12:38PM (#40032362)
You can only monetize garbage for a short time before people catch on and the entire model is destroyed (the good content along with the garbage).
Re:LLMs Are This Close to Destroying the Internet by internetter on Sunday April 14, 2024 @01:43PM (#40032846)
Grifters rarely think ahead, as noted in the article
Re:LLMs Are This Close to Destroying the Internet by internetter on Sunday April 14, 2024 @01:45PM (#40032856)
Yeah, that was my argument. It’s incredibly difficult to find high quality writing at this point (regardless of if it’s for profit or not). The most successful place for me has been hacker news, but that’s not a search engine.
Re:LLMs Are This Close to Destroying the Internet by Kamq on Sunday April 14, 2024 @12:02PM (#40032078)
That seems like it would mean you're restricted to a dataset from before 2022ish or you have to use data that's significantly more expensive to clean.

That seems like it might make the training process untenable.

Re:LLMs Are This Close to Destroying the Internet by coffeebeqn on Sunday April 14, 2024 @12:37PM (#40032354)
Is it really true though? If they scrape HN today I don’t think their sampling would be that “polluted” compared to 2022.

If you were to teach a LLM to play chess today - wouldn’t you use fully AI generated data because that’s by far the best data ?

Re:LLMs Are This Close to Destroying the Internet by johntb86 on Sunday April 14, 2024 @03:40PM (#40033769)
Isn't all of recorded human history up until 2022 enough to train a pretty smart ai? It'll miss out on future trends, but some targeted training on trusted news sources should be enough. Eventually languages will change enough that the old training data will seem dated, but that's going to take a while to happen.
Re:LLMs Are This Close to Destroying the Internet by Kamq on Sunday April 14, 2024 @05:44PM (#40034750)
Yeah, but basically all of that comes from post 2000.

Information production seems to be on an exponential curve, so it won't be that long before you're training on a minority of all data.

Re:LLMs Are This Close to Destroying the Internet by mrspuratic on Sunday April 14, 2024 @12:28PM (#40032269)
The barriers/costs for pushing AI generated noise are so low, I agree there's the potential for it to overwhelm human written content. In a sense it effectively happened to email, again with lots of $doom, but the timescale turned out to be a decade long frog boil. Nobody, or a close approximation to, runs their own email server any more.

In an ironic twist the gatekeepers of the semi-walled gardens/islands of email that arose are now harvesting it for training content. I do so despise those weekly machine generated synopses telling me how I am doing.

Re:LLMs Are This Close to Destroying the Internet by surfingdino on Sunday April 14, 2024 @11:38AM (#40031888)
They are only humans and will face the same issues that humans trying to control the quality of the search engine results did in the past and those policing social media networks do today. They too have admitted they cannot keep up and started using automated tools a long time ago.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @01:13PM (#40032614)
The difference is that there are perverse incentives keeping garbage in search and social. With AI models, the incentive is accuracy.
Re:LLMs Are This Close to Destroying the Internet by surfingdino on Sunday April 14, 2024 @04:20PM (#40034115)
It will change. Sooner or later AI models will be altered to produce output that embeds advertising/paid content/paid messaging.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @04:49PM (#40034350)
There are open source models, so I don't think people are going to be able to get away with that.
Re:LLMs Are This Close to Destroying the Internet by surfingdino on Sunday April 14, 2024 @05:30PM (#40034664)
Money will choose the model. They need lots of expensive compute power so there will have to be a business model and ad-supported is the way to go on the internet.
Re:LLMs Are This Close to Destroying the Internet by Ensorceled on Sunday April 14, 2024 @11:57AM (#40032040)
Your "hahahaha those doomers" comments is either intentionally cruel or Pollyannish. Your "people have agency" "criticism" doesn't address the key points of the article:

1. The damage in the form of the destruction of news sources and replacement of writers with AI has happened and continues to happen. Writers are already jobless. News sources are already dead. But sure, these jobless people with no hope for their industry are "doomers". The article doesn't mention artists, but they are also getting hammered.

2. The worst players already have agency over their data choices, they are using that to build silos and destroy their competitors who don't have that ability.

3. Some social places have already become unusable. Twitter for instance. Threads may be able to push back on this, but they are already a dangerous silo.

Re:LLMs Are This Close to Destroying the Internet by skybrian on Sunday April 14, 2024 @12:17PM (#40032200)
Writing has always been a tough gig and maybe it’s gotten worse, but it doesn’t seem quite that bad? Aren’t some newer writers making money from subscriptions?

From a reader’s point of view, it seems like there’s plenty to read.

Re:LLMs Are This Close to Destroying the Internet by andy99 on Sunday April 14, 2024 @12:54PM (#40032491)
And there are plenty of successful writers too. The internet has just enabled unsuccessful ones to complain about it at scale.
Re:LLMs Are This Close to Destroying the Internet by mrweasel on Sunday April 14, 2024 @01:00PM (#40032520)
News is a really weird place to start with AI, because it obviously doesn't work. The AI cannot go out into the world and do investigating journalism. It still can't access tons of documents that are still hidden away in archives and libraries around the world and it cannot do an sit though endless hours of court room session (even if that might be a fairly good place to start). The only "news" AIs can report on are press releases, that and rehashed articles written by journalists.

Due to the volume of half-arsed news the current AIs can generate your average reader might not notice that some is missing, though I questions that. People are starting to notice that their newspapers doesn't actually have news anymore (and that's just do to cost optimization and competition from ad supported online news).

I fear that we're entering a world where some of us pay for news written by real journalists, while the masses consume garbage "news" which is more tailored to them clicking ads, rather that learning about the world.

Re:LLMs Are This Close to Destroying the Internet by skybrian on Sunday April 14, 2024 @02:25PM (#40033198)
A lot of in-depth news comes from organizations that are funded by paid subscriptions, which gets copied elsewhere. The better websites cite their sources when re-reporting it. So, I guess we can thank the subscribers?

We’re all copying other people’s homework, especially in social media. How much of what you know about the world comes from personal observation? Most people haven’t traveled to most places.

Re:LLMs Are This Close to Destroying the Internet by Ensorceled on Monday April 15, 2024 @09:00AM (#40039999)
> News is a really weird place to start with AI, because it obviously doesn't work. The AI cannot go out into the world and do investigating journalism.

The news business are almost entirely an ad business now. Only a few sites do real investigative journalism and only with a few journalists.

Many news businesses are now owned or majority controlled by billionaires and have the expected editorial biases that also don't really support hard hitting journalism.

Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @01:08PM (#40032583)
1? I don't care. People lose jobs from technological progress, they've been able to see the writing on the wall for a while, and it's still not even a done deal, so frankly if journalists aren't figuring out how to get paid now, it's kind of on them. If their answer to how do I get paid is lawsuits, they deserve the suffering they're going to get.

2. Open source data sets can have curation too, this seems like a silly strawman to make.

3. Twitter isn't just unusable because of AI. It was never more than barely useable to begin with, and many of users didn't help the situation.

Re:LLMs Are This Close to Destroying the Internet by Ensorceled on Monday April 15, 2024 @09:06AM (#40040056)
> 1. I don't care. People lose jobs from technological progress, they've been able to see the writing on the wall for a while, and it's still not even a done deal, so frankly if journalists aren't figuring out how to get paid now, it's kind of on them. If their answer to how do I get paid is lawsuits, they deserve the suffering they're going to get.

I got it from your original post: you don't give a shit. I was just calling you out on it. At least you own it.

> 2. Open source data sets can have curation too, this seems like a silly strawman to make.

It's not a strawman, but "curated open source data sets" are a red herring: those data sets are not what is going to be controlling the online experience of the vast majority of those online.

Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Monday April 15, 2024 @11:44AM (#40042085)
Why should I care about journalists vs anyone else? The pain that journalists suffer will be outweighed by the benefit to society of the tools - assuming people like you don't throw up their hands in defeat and choose to give all their power away to soulless megacorps when choosing which AI tools to use.

News is going to break, and people who break it accurately with personality and a unique take are going to do well regardless if whether newspapers and other bastions of old world journalism continue to exist.

Re:LLMs Are This Close to Destroying the Internet by harborsong on Sunday April 14, 2024 @11:38AM (#40031891)
Maybe it's the whole processing and weighting thing that gives people cause for concern.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @01:14PM (#40032617)
Well then welcome to being a human, because the gatekeepers of knowledge have been doing this since the beginning of time. What do you think the priestly class's original job was?
Re:LLMs Are This Close to Destroying the Internet by marginalia_nu on Sunday April 14, 2024 @12:52PM (#40032478)
Adding to this, the Internet is extremely noisy and polluted with all manner of crap, this has been the status quo for over a decade.

The notion that you could just grab data off the internet completely unvetted, and then train a useful model off that data seems very unrealistic. To get anything useful out of real world crawl data, you need a lot of filtering and processing to select the good bits. This is true if you're training an AI or building a search engine.