News for hackers. Stuff that matters.
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by dTal on Monday April 10, 2023 @10:50AM (#35513011)
I think this is a case of it not having enough capacity to solve such an involved problem in a single pass, despite having all the necessary pieces of abstraction. This can be seen in the way that it can do it with aplomb if you force it to externalize state into its context by making it "show its work" (which is a bit like using a scratch pad) - see sibling comment with dialogue between Einstein and Feynman. I think this further implies that our current feed-forward LMM networks are extremely inefficient, because many recursive algorithms will have been approximated by hard coding the first few steps, and we should expect some recursive variant to be much smaller.

There are no comments on this comment yet.