News for hackers. Stuff that matters.

sillying

Re:Detecting when LLMs are uncertain on Friday October 25, 2024 @02:17PM (#41948001)
I have a simple question. Suppose that to answer a question I can use different phrases, I know the answer but I have several ways to express it. Then a LLM in this case produces tokens with high or low entropy?

Edited several times: I think to avoid this problem the answer of the LLM should be constrained in expression (say Yes or No, fill the blanks, etc). I think in that case we would have a decreasing sequence of the entropy for next token predictions.

Re:Writing a Mathematica Interpreter in Typescript on Tuesday October 22, 2024 @10:25AM (#41914635)
Mathematica has thousand of functions, this seems to be just an algorithm for defining and using rules like that in mathematica.
So it seems that when you have different sources of errors the average of them cancel the noise. I think some property about the sources of error is necessary so in some sense the sources should be independent.