What is an LLM, if not a probabilistic compression scheme?

That’s an interesting rhetorical trick, because answering the question “what is an LLM” requires a lecture series, not a hacker news comment.

Compression schemes take an item, reduce it in size, and return the same item (lossy or lossless) when asked.

LLM's and other generative AI are not designed nor are particularly good at this. What they are good at is returning new results using 'learning' it has developed via training.

If I want to send someone a sample of code, I'll use zip. If I want to generate code from a prompt, I'll use a llm.

> Saying that Generative AI is a compression algorithm shows a lack of understanding of the issues and technology at play.

Because of what?

Because the AI maximalists don’t like it when their favourite toy is reframed into fairly objective terms.

How does a LLM compare to say .ZIP at compressing Romeo and Juliet?

How does ZIP compare to a LLM at answering a prompt to write a short story about cats in the style of Romeo and Juliet?

Because compression and generation are very different concepts.

> Saying that Generative AI is a compression algorithm shows a lack of understanding of the issues and technology at play.

It's actually the other part of the parent comment that's wrong, and a large part of learning something is adding stuff to your dictionary.

Not sure what you mean by the last part, that aside, it veers into introducing language inimical to civility. Both together, makes it also unfair.

I'd argue the exact technology or how it works does not matter at all. The only thing that matters is if it's performed by a human, because making derivative works is a right only granted to humans.

Therefore, by definition and AI cannot create a derivative work. Because it's not human.

I see no reason to grant computer programs human rights. I think a lot of people would have a hard time articulating some reasons to do that. So they don't, and talk about the technology instead. I think that doesn't matter. If you can't tell me, and convince humanity, why a computer program should be granted human rights then I don't think we can even get to a point where the technology itself matters.