GPT-4o's Memory Breakthrough (Needle in a Needlestack)

Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by ben_w on Tuesday May 14, 2024 @05:09PM (#40360190)

I would hope that Byung-Chul Han would not be in the training set (at least not without his permission), given he's still alive and not only is the legal question still open but it's also definitely rude.

This doesn't mean you're wrong, though.

[ Parent | on HN ]

Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by sebzim4500 on Tuesday May 14, 2024 @05:15PM (#40360244)

It's pretty easy to confirm that copywritten material is in the training data. See the NYT lawsuit against OpenAI for example.

[ Parent | on HN ]

Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by ben_w on Tuesday May 14, 2024 @06:14PM (#40360762)

Part of that back-and-forth is the claim "this specific text was copied a lot all over the internet making it show up more in the output", and that means it's not a useful guide to things where one copy was added to The Pile and not removed when training the model.

(Or worse, that Google already had a copy because of Google Books and didn't think "might training on this explode in our face like that thing with the Street View WiFi scanning?")

[ Parent | on HN ]