News for hackers. Stuff that matters.
Re:Ts_zip: Text Compression Using Large Language Models by derefr on Wednesday August 16, 2023 @06:39PM (#37154836)
I think what the GP meant is not that compressed data has intelligence, but rather that a compressor is necessarily "intelligent" in correlation to the degree that it compresses well on arbitrary novel datasets it wasn't tuned for. At least insofar as we define "intelligence" with measures like IQ, that relate to "number of examples of a pattern required to notice the pattern."

To compress data, you have to derive some interesting insight about that data that allows the data to be projected into a lower-dimensional, lower-entropy embedding.

The more things a compressor can do that for — without the compressor itself being made larger — the more "intelligent" the compressor is, no?

Re:Ts_zip: Text Compression Using Large Language Models by tysam_and on Wednesday August 16, 2023 @07:12PM (#37155108)
Yes, that is the point I'm addressing, I'm using Shannon's definition of information and making an argument about the _temporal_ aspects of it. If you're looking for a hint, it's in the Lyapunov exponents of the system, which do not come necessarily from compression alone, though compression I feel is a critical part of the process.

Hence, the set vs superset argument.