Re:There Are No New Ideas in AI Only New Datasets
by hackinthebochs on Monday June 30, 2025 @07:53PM
(#44429176)
You have probably seen examples of LLMs doing the "mirror test", i.e. identifying themselves in screenshots and referring to the screenshot from the first person. That is a genuinely novel question as an "LLM mirror test" wasn't a concept that existed before about a year ago.
[ Parent
| on HN ]
Re:There Are No New Ideas in AI Only New Datasets
by MichaelZuo on Monday June 30, 2025 @08:07PM
(#44429263)
Elephant mirror tests existed, so it doesn’t seem all that novel when the word “elephant” could just be substituted for the word “LLM”?
Re:There Are No New Ideas in AI Only New Datasets
by hackinthebochs on Monday June 30, 2025 @08:43PM
(#44429470)
The question isn't about universal novelty, but whether the prompt/context is novel enough such that the LLM answering competently demonstrates understanding. The claim of parroting is that the dataset contains a near exact duplicate of any prompt and so the LLM demonstrating what appears to be competence is really just memorization. But if an LLM can generalize from an elephant mirror test to an LLM mirror test in an entirely new context (showing pictures and being asked to describe it), that demonstrates sufficient generalization to "understand" the concept of a mirror test.
Re:There Are No New Ideas in AI Only New Datasets
by MichaelZuo on Wednesday July 02, 2025 @05:27PM
(#44449042)
How do you know it’s the one generalizing?
Likely there has been at least one text that already does that for say dolphin mirror tests or chimpanzee mirror teats.