Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by sebzim4500 on Tuesday May 14, 2024 @05:15PM
(#40360244)
It's pretty easy to confirm that copywritten material is in the training data. See the NYT lawsuit against OpenAI for example.
[ Parent
| on HN ]
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack)
by ben_w on Tuesday May 14, 2024 @06:14PM
(#40360762)
Part of that back-and-forth is the claim "this specific text was copied a lot all over the internet making it show up more in the output", and that means it's not a useful guide to things where one copy was added to The Pile and not removed when training the model.
(Or worse, that Google already had a copy because of Google Books and didn't think "might training on this explode in our face like that thing with the Street View WiFi scanning?")