News for hackers. Stuff that matters.
Re:Predict the next frame. Results of training a transformer... by dwaltrip on Tuesday December 05, 2023 @03:09PM (#38536291)
Hypothesis: Intelligence is prediction?
Re:Predict the next frame. Results of training a transformer... by iandanforth on Tuesday December 05, 2023 @03:40PM (#38536622)
You might enjoy 'On Intelligence' by Jeff Hawkins to learn more about this hypothesis. (It's an older book / theory at this point but still worth reading IMHO)
Re:Predict the next frame. Results of training a transformer... by red75prime on Tuesday December 05, 2023 @03:39PM (#38536606)
Yep, prediction of what needs to be done.
Re:Predict the next frame. Results of training a transformer... by stevenhuang on Tuesday December 05, 2023 @06:10PM (#38538234)
Yup, known as predictive coding/free energy principle https://en.wikipedia.org/wiki/Predictive_coding as one of leading theories for human consciousness

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6363942/

Re:Predict the next frame. Results of training a transformer... by romesc on Tuesday December 05, 2023 @07:58PM (#38539176)
Or prediction is intelligence.

I think we need a more rigorous definition to go beyond tautology

Re:Predict the next frame. Results of training a transformer... by keenmaster on Tuesday December 05, 2023 @08:41PM (#38539502)
That intuitively makes sense to me for everything except higher-level goal selection. If someone works at an NGO instead of FAANG because they want to do good for the world, how can that be explained in terms of prediction? If someone minimizes consumption to reduce global warming, how did prediction result in them caring about global warming? I picked those examples on purpose, because they run against some core instincts. Maybe they’re predicting what maximizes peer approval, which optimizes other outcomes too? (Jk :D )
Re:Predict the next frame. Results of training a transformer... by dwaltrip on Tuesday December 05, 2023 @08:47PM (#38539543)
You are right, prediction doesn’t provide one with base motivations or drives. But those seem fundamentally different than raw intelligence.

Your examples aren’t an issue I don’t think. To take one: efforts to minimize one’s consumption to combat global warming could fall in the “help tribe survival” bucket, which makes evolutionary sense as a base drive. And it seems like a reasonably “intelligent” response to that base drive — the predicted results seems positive.

Re:Predict the next frame. Results of training a transformer... by trhway on Tuesday December 05, 2023 @11:08PM (#38540389)
which is very close as to say that intelligence is learning [from mis-prediction], a subject of cybernetics - common machinery across the scales and instances - using a model (brain in case of animals, model/embedding in case of computer, DNA in case of evolution through natural selection) a prediction (which for example in case of evolution is the next generation) is made, and based on the error of that prediction the model is updated for the generation of the next prediction (in case of evolution the DNA of the best fit is the model to be used in the next iteration).