News for hackers. Stuff that matters.

A comment on The Pile is a 825 GiB diverse, open source language modelling data set. Parent comment

Re:The Pile is a 825 GiB diverse, open source language model... by Workaccount2 on Thursday March 07, 2024 @03:44PM (#39634383)
Models are not information archives. The size of the final model is orders of magnitude smaller than the size of the training data.

Somehow people are just not able to get this through their heads. Stable diffusion is like 12GB or something and you have people convinced it's a tool that is cutting and pasting copyrighted works from an enormous image archive.

[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by 7moritz7 on Thursday March 07, 2024 @04:50PM (#39634972)
Stable Diffusion 1.5 is 1.5 to 6 GB depending on the finetune and trained on like 5 billion images
[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by feoren on Thursday March 07, 2024 @05:54PM (#39635638)
> The size of the final model is orders of magnitude smaller than the size of the training data.

Good to know I can avoid copyright on a book just by zipping it up!

[ Parent | on HN ]
Re:The Pile is a 825 GiB diverse, open source language model... by Workaccount2 on Friday March 08, 2024 @09:36AM (#39641409)
No, you can't. But you can by reading the book, writing down the general gist of it (even including some passages), and then storing that.

LLM's are not compressing petabytes of information down to a few gigabytes.

[ Parent | on HN ]