Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode...
by ithkuil on Wednesday May 24, 2023 @02:38AM
(#36054830)
The key idea behind the current wave of AI is that the necessary "structure" will be created through training by sheer amount of data. The idea has been dismissed for decades but it turned out one of the most effective ways to get practical results.
Now, can and should all structure necessarily come from pure training? Is there a better "seed" structure that can make models more effective? Is there some other bits that are missing that are not in the microstructure but in the way the model is connected to the external world and itself (feedback loops etc)?
Now, can and should all structure necessarily come from pure training? Is there a better "seed" structure that can make models more effective? Is there some other bits that are missing that are not in the microstructure but in the way the model is connected to the external world and itself (feedback loops etc)?