News for hackers. Stuff that matters.
Re:Talking About Large Language Models by TeMPOraL on Saturday December 10, 2022 @12:47PM (#33934731)
Average humans? Within 1 stdev from the mean?
Re:Talking About Large Language Models by dragonwriter on Saturday December 10, 2022 @01:43PM (#33935412)
> Average humans? Within 1 stdev from the mean?

This implies that performance has unqiue natural, objective, ratio-level (or at least, a unique consistent interval-level) measure. Otherwise the mean is, itself, meaningless.

“How well you answer a question” doesn't seem to fit that, its maybe at best a (somewhat subjective, still) ordinal quality, so the median (or mode) is the only useful average.

But I think you'll find that without restricting things more than “humans”, both the median and mode of responses to most prompts is... quite bad.