News for hackers. Stuff that matters.
Re:LLMs Are This Close to Destroying the Internet by nicklecompte on Sunday April 14, 2024 @12:20PM (#40032215)
Taken literally this comment suggests that OpenAI et al actually want racist 4chan rants in ChatGPT's training data, or illegal child pornography in DALL-E's training data. Obviously that's not the case: the problem is that it's impossible to effectively audit the amount of data required to make generative AI work.

If Big Tech can't even catch pornography or explicit uses of the n-word, there's no way they'll be able to filter subtle LLM / art generator hallucinations.

Re:LLMs Are This Close to Destroying the Internet by jsheard on Sunday April 14, 2024 @12:31PM (#40032309)
Going one level deeper, even AI-generated training data which doesn't contain hallucinations or other obvious errors could still negatively affect the performance of a new model. GenAI models have a tendency to repeat the same cliches, in sentence structure or the composition of images or whatever else, so even if their output is "fine" in the sense that it's factually correct and everyone has the right number of fingers, a model trained on it will still internalize those cliches and make them even more likely to show up again. That's even harder to audit out because it would only become apparent in aggregate when you realise your new model has learned to interpret "fantasy artwork" as "fantasy artwork in the generic Midjourney RLHF style" since it was trained on so many examples of that.
Re:LLMs Are This Close to Destroying the Internet by wruza on Sunday April 14, 2024 @12:51PM (#40032470)
That's really just a feature of the default system prompt, although ubiquitous. You can character-prompt local LLMs as you like, but no one seems to do that.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @12:54PM (#40032488)
ChatGPT doesn't take character prompts very well, that's why people don't know. Claude on the other hand is a champ with characters.
Re:LLMs Are This Close to Destroying the Internet by CuriouslyC on Sunday April 14, 2024 @12:43PM (#40032394)
Not so much that, as OpenAI has finite resources and is allocating them to areas that are more valuable.
Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @12:42PM (#40032383)
Has it been demonstrated that there is illegal CSAM in the DALL-E training?
Re:LLMs Are This Close to Destroying the Internet by jsheard on Sunday April 14, 2024 @12:49PM (#40032451)
I don't think it has, if only because OpenAI aren't transparent about where they get their image training data from, but the most widely used public image generation dataset was found to contain CSAM: https://www.theverge.com/2023/12/20/24009418/generative-ai-i...
Re:LLMs Are This Close to Destroying the Internet by nicklecompte on Sunday April 14, 2024 @01:24PM (#40032688)
It hasn't been demonstrated in DALL-E, but this was a problem in older OpenAI text-video models, and it's in plenty of CommonCrawl/etc data. There is anecdata from contractors suggesting that DALL-E generated CSAM imagery during RLHF, which is circumstantial evidence that similar images are somewhere in the training data. I think it's fair to say we can't rely on OpenAI's technical competence to remove this stuff from their training data, despite claims to the contrary w.r.t DALL-E 2. (Likewise it seems Copilot Designer churns out CSAM like candy.)

Your attitude is naive and competely at odds with (honest) AI safety research. The rational thing for users is to assume DALL-E 2 is contaminated by CSAM unless OpenAI can publicly demonstrate otherwise. Their leadership is too stupid and dishonest for people to take their word.

https://ieeexplore.ieee.org/abstract/document/9423393

https://arxiv.org/pdf/2110.01963.pdf?trk=public_post_comment...

https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...

Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @03:07PM (#40033519)
You're saying my attitude is "naive". I'm saying that it's irresponsible for you to state that there is CSAM in DALL-E's training data unless you can back that up.

I find OpenAI's refusal to document what's in the training data infuriating, but I that doesn't lead me to assume they weren't able to filter out CSAM from it.

If they don't care about this issue at all, why are they exposing low paid workers in Kenya to these conditions?

Re:LLMs Are This Close to Destroying the Internet by candiodari on Sunday April 14, 2024 @12:46PM (#40032422)
Why would you need to?

Obviously generalizing over pictures of humans and pictures of porn will enable CSAM generation ...

Plus a LOT of things have been declared illegal pornographic material in various places: all porn (China), drawn CSAM, any kind of drawn porn with unclear ages, ...

Re:LLMs Are This Close to Destroying the Internet by simonw on Sunday April 14, 2024 @01:01PM (#40032521)
> "Why would you need to?"

If someone is going to suggest that there is "illegal child pornography in DALL-E's training data" I think it needs to be backed up - that's a pretty big accusation.