News for hackers. Stuff that matters.

A comment on You can't trust the AI hype. Parent comment

Re:You can't trust the AI hype by sharemywin on Tuesday August 22, 2023 @04:24PM (#37228054)
The thing I find most interesting about LLMs is the emergent skills that happen almost instantly(based on model size) that weren't inherently trained for.

Picking movie names from emojis seems like something other than interpolation.

https://www.assemblyai.com/blog/emergent-abilities-of-large-....

[ Parent | on HN ]
Re:You can't trust the AI hype by scottmas on Tuesday August 22, 2023 @05:28PM (#37228755)
Not really IMO. At scale the model learns that the rainbow emoji is literally a synonym for “rainbow”. Computers are great at memorizing giant lists of facts such as this. It feels way more amazing for an LLM to guess the Carebear Movie from the rainbow emoji and bear emoji but it’s literally the same as asking the LLM to guess a movie that involves rainbows and bears with words.

It’s just an embedding space buried within the bowls of the LLM.

[ Parent | on HN ]