Re:Use Prolog to improve LLM's reasoning
by shchegrikovich on Friday October 18, 2024 @12:12PM
(#41880757)
I have another example - just a few people believed that you can apply 'a simple next token prediction algorithm' and achieve what we know as LLM. From my perspective, in the past few years, we've tried a lot of different approaches to improve LLM reasoning; some of them were good, others not so good. We need to keep trying and researching. 'Prolog + LLM' is not the answer to all questions, but it looks like a good step to move us forward.
Re:Use Prolog to improve LLM's reasoning
by sgdfhijfgsdfgds on Friday October 18, 2024 @12:22PM
(#41880879)
> 'Prolog + LLM' is not the answer to all questions, but it looks like a good step to move us forward.
Or it's a thing people can write papers about, and chase reproducibility on afterwards, as the shell game of claiming LLM reasoning continues.
Prolog is actually pretty difficult to do right, even if you are skilled. It actually requires reasoning. And many of the examples in the training set will be wrong or be full of backtracking that is itself extremely difficult for a person to comprehend at a glance; why should an LLM be better at using it? There can't even be that much data in the training set.
Ultimately, though: stop believing in magical solutions to fundamental problems. This is nuts.