All of this seems possible in a deterministic system which evaluates whether or not to retain a piece of information based on past experience.

LLMs can be deterministic too at temperature 0

But they don’t work very well at temp 0. Better to seed your random numbers.

Food for thought: randomness is in the eye of the beholder. It’s only random to you if you don’t know how to predict it. So can a mind ever truly be stochastic? Perhaps to the minds of others, but never to itself.

No, it's random if it can't be perfectly predicted with perfect knowledge.

> It is undeniable that human reasoning is a stochastic process

It can still be a deterministic process. If anything came out of the whole LLM story for me it is that I am even more convinced that it is.

My (somewhat educated, but still naive) idea why it looks like a stochastic process is that the brain gets incredible amounts of random input. We literally get bombarded with particles and energy ever instance we live, from photons hitting out retinas , molecules transferring "heat" energy into our skin to sound waves hitting our ear drums.

Ever noticed that you get more productive when you get up from your screen? Or how some people work better while listening to music? How you find your answer just as you start to explain it to a colleague?

I would argue that this is due to a limited "entropy" pool available to the brain. Just changing the input to the system replenishes the pool.

According to physics, it must be stochastic. The only question is distribution, and whether we can ignore outlier cases.

Are you saying this because quantum mechanics is random in some interpretations and is our most fundamental theory?

That’s a bit of a stretch when we don’t actually know if QM is objectively random, but it could be sure. But then what about things like, is it random that 1+1=2. No…what are you really saying is random when a human answers this? I think even if you make the assumption QM is objectively random, thought is the hard problem after all and we might not want to jump so far head. Math certainly isn’t random and we can think about it.

Brains are complex probability machines but doesn’t work only on language. It interacts with and learns from the environment, sets its own objective functions and tries to learn better.