News for hackers. Stuff that matters.
Re:Teaching Large Language Models to Self-Debug by gopalv on Wednesday April 12, 2023 @11:34PM (#35550596)
> and >90 after that

This is such a circular thing, that I feel like it is amazing to see it.

The reason LLMs use a NN is because they're trying to encode a probability function for generating the passage.

And now, you are encoding another n-gram follower exercise (i.e 1+1 = 2) on top of it :)

Re:Teaching Large Language Models to Self-Debug by Paul-Craft on Thursday April 13, 2023 @06:27AM (#35553373)
Yeah... and I'm kind of suspicious of the whole "without changing the weights" deal, because adding working context to the model, like telling it the algorithm for adding numbers really sounds like there's some model state that's getting mutated, even if it's not stored in a file called weights.dat or whatev.