The Pile is a 825 GiB diverse, open source language modelling data set

Re:The Pile is a 825 GiB diverse, open source language model... by numpad0 on Thursday March 07, 2024 @06:03PM (#39635715)

> you get something like 1.2 bits per image, and it is physically impossible to get this kind of a compression ratio.

IIRC it was like 1.4 bytes before adding in random initial and prompt. And Amiga Four-byte Burger is 4 bytes long.

[ Parent | on HN ]

Re:The Pile is a 825 GiB diverse, open source language model... by bayindirh on Thursday March 07, 2024 @06:07PM (#39635748)

For reference, this [0] is Amiga Four Byte Burger, and it looks impossibly good.

[0]: https://bytecellar.com/2023/04/24/lost-amiga-four-byte-burge...

[ Parent | on HN ]

Re:The Pile is a 825 GiB diverse, open source language model... by fennecfoxy on Friday March 08, 2024 @05:57AM (#39639913)

I don't understand how this is relevant? It's not generated from 4 bytes of data or code, surely the name of the piece is just a joke about byte = bite?

[ Parent | on HN ]