News for hackers. Stuff that matters.
Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @04:49PM (#43475801)
The correlations are 0.25-0.5, which is quite poor (Gaussian distribution plots with those correlations look like noise). That's before analyzing the methodology and assumptions.
Re:Using LLMs to Understand Language Processing in the Human... by bravura on Tuesday March 25, 2025 @05:24PM (#43476145)
Correlation of 0.25-0.5 being poor is very problem dependent.

For example, in difficult perceptual tasks ("can you taste which of these three biscuits is different" [one biscuit is made with slightly less sugar]), a correlation of 0.3 is commonplace and considered an appropriate amount of annotator agreement to make decisions.

Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @05:48PM (#43476373)
Yes for certain things like statistical trading (assuming some kind of "nice" Gaussian-like distribution) where you have lots of trades and just need to be more right than wrong it's probably useful.

Not here though, where you are trying to prove a (near) equivalence.

Re:Using LLMs to Understand Language Processing in the Human... by ogogmad on Tuesday March 25, 2025 @04:50PM (#43475812)
Isn't correlation a wrong way of measuring statistical dependence anyway? I thought that entropy-based method like Relative Entropy did this better.
Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @04:58PM (#43475896)
Yeah, Taleb has some pretty convincing arguments that correlation is generally bad/misleading and entropy is better.