Re:Language models and linguistic theories beyond words
by jonahrd on Thursday November 16, 2023 @12:15AM
(#38285992)
This is super interesting. Are there more examples I can see? The one in the article is a famous song which makes me wonder if it's really "decompressing" the data, or just being hunted towards a very common popular pattern of tokens.
https://www.piratewires.com/p/compression-prompts-gpt-hidden...