* * *

I definitely have that question too. I view us as big LLMs.

But, even if we drop that interesting edge case i suspect we can make something very useful with the primitive that LLMs offer.. in the calculator example. ChainLang and co seem a really interesting tool for LLMs.

And additionally, whether our memory and long term learning - and even our goal-choosing - is fundamentally different from an indexed storage of strings of tokens that can be brought back into short-term context when “triggered” by their embedding-similarity to the current context.

The question is what happens when you go multimodal (which these things can do) and GPT(N+1) learns the associations between words and images/video, as well as the relationships between successive frames of video, at what point does it become unreasonable to claim that it doesn't "understand" something? How good at general-purpose predicting does an AI have to be in order for people to accept that it obviously has an internal model of things and is capable of abstractions?

(Assuming that this happens, of course. Diminishing returns could make scaling infeasible past some point, for instance.)