Re:The bigger-is-better approach to AI is running out of road
by ruszki on Sunday June 25, 2023 @07:51AM
(#36467248)
According to physics, it must be stochastic. The only question is distribution, and whether we can ignore outlier cases.
Re:The bigger-is-better approach to AI is running out of road
by meroes on Sunday June 25, 2023 @08:43AM
(#36467615)
Are you saying this because quantum mechanics is random in some interpretations and is our most fundamental theory?
That’s a bit of a stretch when we don’t actually know if QM is objectively random, but it could be sure. But then what about things like, is it random that 1+1=2. No…what are you really saying is random when a human answers this? I think even if you make the assumption QM is objectively random, thought is the hard problem after all and we might not want to jump so far head. Math certainly isn’t random and we can think about it.
It can still be a deterministic process. If anything came out of the whole LLM story for me it is that I am even more convinced that it is.
My (somewhat educated, but still naive) idea why it looks like a stochastic process is that the brain gets incredible amounts of random input. We literally get bombarded with particles and energy ever instance we live, from photons hitting out retinas , molecules transferring "heat" energy into our skin to sound waves hitting our ear drums.
Ever noticed that you get more productive when you get up from your screen? Or how some people work better while listening to music? How you find your answer just as you start to explain it to a colleague?
I would argue that this is due to a limited "entropy" pool available to the brain. Just changing the input to the system replenishes the pool.