Finding bugs in some models doesn’t mean you have a point about intelligence. If somebody could apply a similar argument to dismiss human intelligence, you don’t have a point. And here it goes: the most advanced human intelligence can’t reliably multiply large numbers or recall digits of Pi. Obviously humans are dumber than pocket calculators.

The things that LLM’s are bad at are largely solved problems using much simpler technology. There is no reason that LLM’s have to be the only component in an intelligent agent. Biological brains have Specialized structures for specialized tasks like arithmetic. The solution is probably integration of LLMs as a part of a composite system that includes database storage, a code execution environment, and multiple agents to form a goal directed posit - evaluate loop.

I’ve had pretty remarkable success with this architecture running on 12b models and I’m a nobody with no resources.

LLM’s by themselves just come up with the first thing that crosses their”mind”. It shouldn’t be surprising that the very first unfiltered guess about a solution might be suboptimal.

There is a vast amount of knowledge embedded in our cultural matrix, and a lot of that is captured in the common crawl and other datasets.llms are like a search engine for that data , based on meaning rather than semantics.

Some of the most advanced AI are tool users and can both write and crucially also execute python, and embed the output in their responses.

> or even solve really simple logic puzzles for little kids.

As given in a recent discussion: https://chatgpt.com/share/ee013797-a55c-4685-8f2b-87f1b455b4...

(Custom instructions, in case you're surprised by the opening of the response).