News for hackers. Stuff that matters.
Re:The Case That A.I. Is Thinking by procaryote on Wednesday November 05, 2025 @02:21AM (#45820289)
I'm not sure why you'd describe "sampling from a probability distribution over possible tokens" as "minimize surprise in predictions" other than to make it sound similar to the free energy thing.

The free energy thing as I understand it has internal state, makes predictions, evaluates against new input and adjusts it internal state to continuously learn to predict new input better. This might if you squint look similar to training a neural network, although the mechanisms are different, but it's very distinct from the inference step

Re:The Case That A.I. Is Thinking by FloorEgg on Wednesday November 05, 2025 @03:06AM (#45820565)
"Minimize surprise" and "maximize accurate predictions" are the same thing mathematically. Minimize free energy = minimize prediction error.

LLMs do everything modelled in the free energy principle, they just don't do continuous learning. (They don't do perceptual inference after RL)

Your tone ("free energy thing" and "if you squint") comes off as dismissive and not intellectually honest. Here I thought you were actually curious, but I guess not?

Re:The Case That A.I. Is Thinking by procaryote on Wednesday November 05, 2025 @12:40PM (#45825615)
Poor wording on my side, I'm sorry. Thank you for explaining your reasoning
Re:The Case That A.I. Is Thinking by FloorEgg on Wednesday November 05, 2025 @05:17PM (#45828861)
Thank you for saying that :)