News for hackers. Stuff that matters.
Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl... by klipt on Saturday April 05, 2025 @01:14PM (#43594975)
> LLMs learn from examples where the logits are not probabilities, but how a given sentence continues (only one token is set to 1).

But enough data implies probabilities. Consider 2 sentences:

"For breakfast I had oats"

"For breakfast I had eggs"

Training on this data, how do you complete "For breakfast I had..."?

There is no best deterministic answer. The best answer is a 50/50 probability distribution over "oats" and "eggs"

Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl... by ksec on Saturday April 05, 2025 @02:28PM (#43595528)
So it is still largely, probabilities pattern matching?
Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl... by klipt on Sunday April 06, 2025 @12:15PM (#43602576)
You can model the whole universe with probabilities!