News for hackers. Stuff that matters.

A comment on The Pile is a 825 GiB diverse, open source language modelling data set. Parent comment

Re:The Pile is a 825 GiB diverse, open source language model... by layer8 on Thursday March 07, 2024 @03:02PM (#39633941)
Summary books for example are legal, so there is some threshold of compression where things are fine.
[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by knodi123 on Thursday March 07, 2024 @03:19PM (#39634126)
are you referring to things like CliffNotes ?
[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by layer8 on Thursday March 07, 2024 @03:26PM (#39634186)
I’m referring to the “Summary of <some other book>” booklets you can find on Amazon. Also services like Blinkist.
[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by knodi123 on Thursday March 07, 2024 @07:42PM (#39636421)
Huh. Never heard of those. The internet sure has a lot of dark little nooks and crannies.
[ Parent | on HN ]