News for hackers. Stuff that matters.
Re:Are Emergent Abilities of Large Language Models a Mirage? by p-e-w on Monday May 01, 2023 @02:23AM (#35769628)
Calling human intuition "just another metric" ignores the fact that human intuition performs spectacularly well at many high-level tasks.

Whatever "insight" and "understanding" actually mean, there is no denying that they are immensely useful, which is why we want AIs that can replicate them.

When trying to understand complex systems that don't yield to quantitative analysis in an obvious way, the starting point should be to assume that the intuitive evaluation is (roughly) correct, until proven otherwise. Trying to cast this intuition into a simple metric and then using that metric (or other simple metrics) to demonstrate that the intuition is wrong is circular reasoning.

Re:Are Emergent Abilities of Large Language Models a Mirage? by kelseyfrog on Monday May 01, 2023 @03:13AM (#35769906)
> human intuition performs spectacularly well at many high-level tasks.

It's also the number one most common cognitive bias. Humans are especially prone to reification - the confusion that the construction of a measure equates to an objective reality.

Humans are often launder subjectivity through the creation of metrics without 1) knowing they've done so, and 2) become emotional when accused of having done so.