Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode...
by TeMPOraL on Wednesday May 24, 2023 @08:13PM
(#36065632)
Yup. I totally expect you'll be able to eke out significant performance boost if you chain up the LLMs, so that e.g. you feed the initial query to a first-stage GPT-4 several times (likely in parallel), feed those to some kind of filter models that pass or reject the output, looping until you have, say, 3 passing outputs, then feed that to a summarizer, etc. Maybe play with generating system prompts so that you have multiple entirely different takes on the same query, or stage it. Or, you know, have GPT-4 look at the query and propose a graph of subsequent invocations for you.
I wish I had time to play with it some more right now. The pace of progress in the field is giving me a serious case of FOMO.
The output of LLMs is kind of like our stream of consciousness, there's a lot of things I think, then discount after internally reflecting on the thought which the often leads to a more correct solution. Having an LLM 'think' like this natively would massively increase the necessary the amount of compute needed, hence the expense, so at least in any public products it's not being done at this time.