News for hackers. Stuff that matters.

A comment on Authors Seek Meta's Torrent Client Logs and Seeding Data in AI Piracy Probe. Parent comment

Re:Authors Seek Meta's Torrent Client Logs and Seeding Data ... by ben_w on Monday January 20, 2025 @06:06PM (#42774213)
IMO, if the AI were more sample-efficient (a long-standing problem that predates LLMs), they would be able to learn from purely open-licensed content, which I think Wikipedia (CC-BY-SA) would be an example of? I think they'd even pass the share-alike requirements, given Meta are giving away the model weights?

https://en.wikipedia.org/wiki/Wikipedia:Copyrights

[ Parent | on HN ]
Re:Authors Seek Meta's Torrent Client Logs and Seeding Data ... by wizzwizz4 on Monday January 20, 2025 @07:20PM (#42774789)
Since this is Wikipedia, it could even satisfy the attribution requirements (though most CC-licensed corpora require attributing the individual authors).
[ Parent | on HN ]
Re:Authors Seek Meta's Torrent Client Logs and Seeding Data ... by visarga on Monday January 20, 2025 @11:31PM (#42776640)
Alteratively if they trained the model on synthetic data, filtered to avoid duplication, then no copyrighted material would be seen by the model. For example turn an article into QA pairs, or summarize across multiple sources of text.
[ Parent | on HN ]
Re:Authors Seek Meta's Torrent Client Logs and Seeding Data ... by thesz on Monday January 20, 2025 @11:47PM (#42776730) 

  > trained the model on synthetic data
You get knowledge collapse [1] this way.

[1] https://arxiv.org/abs/2404.03502

[ Parent | on HN ]
Re:Authors Seek Meta's Torrent Client Logs and Seeding Data ... by ben_w on Tuesday January 21, 2025 @08:14AM (#42779746)
You can get knowledge collapse.

But based on how often people were already doing this with buggy human-written software[0][1] — and even though there are counter-examples (e.g. learning Go from self-play based only on the rules) — IMO most people don't think about this in the right way and will therefore treat these things as magical oracles when they shouldn't.

No silver bullets.

[0] https://en.wikipedia.org/wiki/British_Post_Office_scandal

[1] https://en.wikipedia.org/wiki/Computer_says_no

[ Parent | on HN ]