News for hackers. Stuff that matters.

A comment on Ask HN: What can I personally do to fight AGAINST the Copilot class action?. Parent comment

Re:Ask HN: What can I personally do to fight AGAINST the Cop... by visarga on Friday November 04, 2022 @02:07PM (#33471275)
The dataset is so large they only do one single pass, and the content is deduplicated. A study showed that it's usually duplicated content that is overfitted and later regurgitated. Examples that are repeated in many places might have slight variations that help them pass the filter.
[ Parent | on HN ]
Re:Ask HN: What can I personally do to fight AGAINST the Cop... by Kim_Bruning on Friday November 04, 2022 @03:32PM (#33472481)
Interesting!

Do you have a link(s) to the study, or more details on training?

[ Parent | on HN ]
Re:Ask HN: What can I personally do to fight AGAINST the Cop... by visarga on Friday November 04, 2022 @04:08PM (#33473032)
Deduplicating Training Data Makes Language Models Better

https://arxiv.org/pdf/2107.06499

Quantifying Memorization Across Neural Language Models

https://arxiv.org/pdf/2202.07646.pdf

[ Parent | on HN ]