Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C...
by jkingsman on Tuesday December 31, 2024 @02:53AM
(#42557231)
How does one impart textual knowledge discovered by humans without language?
[ Parent
| on HN ]
Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C...
by thelittleone on Tuesday December 31, 2024 @04:57AM
(#42557681)
Couldn't we use an AI model trained on historical text data (up to today) to predict likely events for tomorrow? Taking this further, a sufficiently advanced AI system could potentially analyze human-generated text up to any given point in history to understand patterns of human thought and behavior, then project those patterns forward. This speaks to your point about human language - while we need text data for initial training, the AI's internal representations and predictions could potentially transcend human language constraints.
Re:Coconut by Meta AI – Better LLM Reasoning with Chain of C...
by viraptor on Tuesday December 31, 2024 @06:29AM
(#42558080)
The training of the LLM itself would still use the human language. But you could add an extra channel that's never given any text or direct dataset training. Keep it purely a connection between hidden layers of different instances of LLM and train using the usual loss of perplexity or similar metric.
The interesting thing then would be - does it converge to similar embedding space as the input, or can LLMs create a more efficient "language".