News for hackers. Stuff that matters.
Re:What does it mean when an LLM “hallucinates” & why do LLM... by chrisjj on Tuesday August 22, 2023 @02:25PM (#37226704)
> It's reasonable to call minimising the training loss function the "objective" of the LLM

This article isn't doing that, though. What it is calling the model objective is "producing clear and coherent text".

> Whether "producing clear and coherent text" is a good characterisation of a "predict the next token" loss function is another question though.

I agree. But lets us be clear that prediction too is a misnomer. Simply generating a word coherent with prior words is not prediction.

There are no comments on this comment yet.