Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl...
by fragmede on Saturday April 05, 2025 @02:22PM
(#43595466)
The combinatorics on choosing 500 pieces (words) out of a bag of 1.8 million pieces (approx parameters per layer for GPT-3) with replacement, and order matters works out to be something like 10^4600. Maybe you can't call that creativity, but you've got to admit that's a pretty big number.
[ Parent
| on HN ]
Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl...
by belter on Saturday April 05, 2025 @02:29PM
(#43595543)
I said No handwaving with scale. :-)
Re:Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearl...
by fragmede on Saturday April 05, 2025 @02:55PM
(#43595861)
Right—but why should “new ABS plastic” be the bar for creativity? If the kid builds a structure no one’s ever imagined, from an unimaginably large box of Lego, isn’t that still novel? Sure, it’s made from known parts—but so is language. So are most human ideas.
The demand for outputs that are provably untraceable to training data feels like asking for magic, not creativity. Even Gödel didn’t require “never seen before atoms” to demonstrate emergence.