Re:Ts_zip: Text Compression Using Large Language Models
by tysam_and on Wednesday August 16, 2023 @07:12PM
(#37155108)
Yes, that is the point I'm addressing, I'm using Shannon's definition of information and making an argument about the _temporal_ aspects of it. If you're looking for a hint, it's in the Lyapunov exponents of the system, which do not come necessarily from compression alone, though compression I feel is a critical part of the process.
Hence, the set vs superset argument.
To compress data, you have to derive some interesting insight about that data that allows the data to be projected into a lower-dimensional, lower-entropy embedding.
The more things a compressor can do that for — without the compressor itself being made larger — the more "intelligent" the compressor is, no?