News for hackers. Stuff that matters.
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by Eisenstein on Tuesday May 14, 2024 @05:21PM (#40360294)
My idea is to buy to a unpublished novel or screenplay with a detailed, internally consistent world built in to it and a cast of characters that have well crafted motivations and then ask it to continue writing from an arbitrary post-mid-point by creating a new plot line that combines two characters that haven't yet met in the story. If it understands the context it should be able to write a new part of the story and will be able to use a reader's intuitive sense of the character's motivations to move through their arc.

This whole thing would have to be kept under lock-and-key in order to be useful, so it would only serve as a kind of personal benchmark. Or it could possibly be a prestige award that is valued for its conclusions and not for its ability to use the methodology to create improvements in the field.

Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by visarga on Tuesday May 14, 2024 @05:48PM (#40360544)
You can only use it for a short while, they get a copy as well.
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by Eisenstein on Tuesday May 14, 2024 @06:19PM (#40360809)
I have been thinking about this for use in evaluating locally run models, so I didn't make that connection in this case. I guess it would have limited utility.
Re:GPT-4o's Memory Breakthrough (Needle in a Needlestack) by semi-extrinsic on Wednesday May 15, 2024 @02:04AM (#40363583)
Just use memes. People generate new high-quality niche memes so fast it's impossible for the LLMs to keep up.