News for hackers. Stuff that matters.
Re:Generative AI's crippling failure to induce robust models... by etaioinshrdlu on Sunday June 29, 2025 @12:53AM (#44410445)
I don't think it's accurate anymore to say LLMs are just really good word predictors. Especially in the last year, they are trained with reinforcement learning to solve specific problems. They are functions that predict next tokens, but the function they are trained to approximate doesn't have to be just plain internet text.
Re:Generative AI's crippling failure to induce robust models... by voidhorse on Sunday June 29, 2025 @01:03AM (#44410491)
Yeah, that's fair. It's probably more accurate to call them sequence predictors or general data predictors than to limit it to words (unless we mean words in the broad, mathematical sense) they are free monoid emulators
Re:Generative AI's crippling failure to induce robust models... by antonvs on Sunday June 29, 2025 @03:41AM (#44411049)
And what are humans?
Re:Generative AI's crippling failure to induce robust models... by sgt101 on Sunday June 29, 2025 @05:48AM (#44411694)
Humans are humans - to deny that we are thinking, reasoning, living beings is a strange thing to do.

You can taste a beer, laugh so much it hurts, come to know how something works.

Re:Generative AI's crippling failure to induce robust models... by antonvs on Tuesday July 01, 2025 @11:17AM (#44434757)
I didn’t deny anything.

The parent comments were attempting to characterize LLMs as something more general than “word predictors”. The alternative “sequence predictors” was proposed.

My question relates to whether we have any reason to believe that the relevant aspects of human cognition are anything more than that.

Certainly humans have some advantages, like the ability to continuously learn (although there’s very strong evidence that we have a pretraining phase too, for example the difficulty of learning new languages as an adult vs. as a child.) But fundamentally, it’s not clear to me that our own language production skills aren’t “just” sequence prediction.

Perhaps, as the OP article speculates, there are other important components, like “models of the world”. But in that case, it may be that we’re augmented sequence predictors.