News for hackers. Stuff that matters.

A comment on Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book.

Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by deafpolygon on Sunday June 15, 2025 @08:50AM (#44282054)
It will generate a correct next token 42% of the time when prompted with a 50 token quote.

Not 42% of the book.

It's a pretty big distinction.

[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by deviation on Sunday June 15, 2025 @09:13AM (#44282146)
A... massive distinction.
[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by asplake on Sunday June 15, 2025 @09:34AM (#44282215)
“… well enough to reproduce 50-token excerpts at least half the time”
[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by chiph2o on Sunday June 15, 2025 @02:12PM (#44283963)
This means that if we start with 50% of the book then there is 42% chance that we can recreate the remaining 50%.

What is the distinction between understanding and memorization? What is the chance that understanding results in memorization (may be in case of humans)?

[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by ipaddr on Monday June 16, 2025 @01:50AM (#44286926)
It stores how often characters will come next based on how often they happen in copyright material. It can reproduce parts because those values are a fingerprint.

It should break copyright laws as written now but too much money involved.

[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by j16sdiz on Sunday June 15, 2025 @11:27PM (#44286414)
next _50_ tokens 42% of the time

not just next token.

This is like: tell it a random sentence in the book, it will give you the next sentence 42% of time.

[ Parent | on HN ]
Re:Meta's Llama 3.1 can recall 42 percent of the first Harry... by amlib on Monday June 16, 2025 @05:26PM (#44293530)
That said, I bet that if you could lower the inference temperature such chances would improve by a lot.
[ Parent | on HN ]