>Perhaps we will use a LLM to interpret the secret thoughts of another LLM?
this is a pretty active area of research with sparse autoencoders
Re:Learning to Reason with LLMs
by navigate8310 on Thursday September 12, 2024 @06:16PM
(#41526211)
At this point the G in GPU must be completely dropped
Gen-ai Processing Unit
Re:Learning to Reason with LLMs
by hobo_in_library on Thursday September 12, 2024 @06:13PM
(#41526187)
"...either way we're going to need more GPUs." posted the LLM, rubbing it's virtual hands, cackling with delight as it prodded the humans to give it MOAR BRAINS
> Now that seems less likely. At least OpenAI can see what it's thinking.
When it's fully commercialized no one will be able to read through all chains of thoughts and with possibility of fine-tuning AI can learn to evade whatever tools openai will invent to flag concerning chains of thoughts if they interfere with providing the answer in some finetuning environment.
Also at some point for the sake of efficiency and response quality they might migrate from chain of thought consisting of tokens into chain of thought consisting of full output network states and part of the network would have dedicated inputs for reading them.
Now that seems less likely. At least OpenAI can see what it's thinking.
A next step might be allowing the LLM to include non-text-based vectors in its internal thoughts, and then do all internal reasoning with raw vectors. Then the LLMs will have truly private thoughts in their own internal language. Perhaps we will use a LLM to interpret the secret thoughts of another LLM?
This could be good or bad, but either way we're going to need more GPUs.