Re:Talking About Large Language Models
by TeMPOraL on Saturday December 10, 2022 @12:47PM
(#33934731)
Average humans? Within 1 stdev from the mean?
[ Parent
| on HN ]
Re:Talking About Large Language Models
by dragonwriter on Saturday December 10, 2022 @01:43PM
(#33935412)
> Average humans? Within 1 stdev from the mean?
This implies that performance has unqiue natural, objective, ratio-level (or at least, a unique consistent interval-level) measure. Otherwise the mean is, itself, meaningless.
“How well you answer a question” doesn't seem to fit that, its maybe at best a (somewhat subjective, still) ordinal quality, so the median (or mode) is the only useful average.
But I think you'll find that without restricting things more than “humans”, both the median and mode of responses to most prompts is... quite bad.