Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode...
by pixl97 on Wednesday May 24, 2023 @12:06PM
(#36060044)
>pulling in the proper things, and giving them the correct amount of attention.
This is likely one of the harder problems to solve. One of the strengths humans seem to have is the power of analogy. These analogies quite often can lead us into new paths of thought, or at least in the correct direction.
How do you do this in a machine system without wasting tons of power chasing dead ends, not sure?
It seems to me that LLMs are perhaps capable of making up the inner thought loop, and the rest of the obvious systems seem doable. The context seems to be the hard problem; pulling in the proper things, and giving them the correct amount of attention.