And there comes the fact LLMs are actually terrible at coding beyond advanced regurgitation, precisely because their single-pass architecture does not give them Turing completeness to model a Turing machine, which may also be the reason they model the full chain of cause and effect to formulate a coherent plan...
Re:The "Chain of Thought" Delusions
by FeepingCreature on Monday May 13, 2024 @03:31AM
(#40340621)
Turing machines are an irrelevancy for programming, and LLMs can code ... not well, but at a higher level than regurgitation. ("Advanced" regurgitation can, of course, describe anything.)
There are people working on that! Google for LLM + PDDL.
There are big problems with hallucinations because LLMs are not smart enough to know when they're starting to make mistakes.
But there's lots of work in this area, and generally in different ways to nail neural and symbolic systems together.
Have they tried that?