Given that most people have computers and RAM is dirt cheap right now, I'm already maxing out all of my devices.
But it does seem like a single call to an LLM is not the correct approach.
I’ve gotten the best results from attempting to mirror how I would operate in response to a prompt. Much of this is colored by decades of philosophical readings, which in my case means a decent amount of analytical philosophy along with latter Wittgenstein and other post linguistic turn thinkers.
So when I break things down, either in some kind of approach inspired by Frege, Wittgenstein or unadorned self-reflection, I can see that the unconscious mind seems to be doing something recursive before my conscious mind is ready to start fumbling over the word.
It seems like an LLM is just part of a larger cognitive architecture that will use these metacognitive approaches.
It’s also very obvious that mammalian brains are much more efficient than GPUs.
First, you'll notice the added "work it out step by step" to encourage what is called chain-of-thought in the literature. Second, generate say, five responses from the LLM, then take all five responses and uses them in another prompt where you ask the LLM to reflect on all of the five responses, a process being called reflection. Then you take all of that, including the original prompts, and in another separate prompt ask it to figure out the best response, a process known as resolving. You could end it with another prompt to remove the step-by-step work and just return a succinct answer.
Credit to https://www.youtube.com/watch?v=wVzuvf9D9BU