News for hackers. Stuff that matters.
Re:Recursively Summarizing Enables Long-Term Dialogue Memory... by roseway4 on Saturday September 02, 2023 @08:34PM (#37366696)
I’m struggling to understand what’s novel here. LLM-based summarization of chat history memory is a well-established technique implemented by many LLM frameworks. Summarizing on every message is, as proposed in the paper, a major performance bottleneck and adds significant latency to the chat loop.

Many implementations utilize a fixed sized buffer, progressively summarizing batches of older memories when they fall out of the buffer. Ideally, this is also done out of band to the chat loop.

I’m an author of Zep[0], an open source long-term memory store, and this is how we implemented summarization.

0: https://github.com/getzep/zep

Re:Recursively Summarizing Enables Long-Term Dialogue Memory... by anotherpaulg on Saturday September 02, 2023 @09:09PM (#37366843)
Aider does this too, using a background thread to summarize the messages older than the last N.

https://github.com/paul-gauthier/aider/blob/main/aider/histo...

Re:Recursively Summarizing Enables Long-Term Dialogue Memory... by px43 on Saturday September 02, 2023 @10:11PM (#37367129)
Yeah, I'm pretty novice, but I took that hourish long Andrew Ng class on LangChain, and they covered recursive summarization as a standard memory management technique.

https://www.deeplearning.ai/short-courses/langchain-for-llm-...

Re:Recursively Summarizing Enables Long-Term Dialogue Memory... by LightHugger on Tuesday September 05, 2023 @09:34AM (#37391437)
This poster is a nazi. Just thought I'd make sure everyone knows.
Re:Recursively Summarizing Enables Long-Term Dialogue Memory... by chrgy on Saturday September 02, 2023 @10:56PM (#37367325)
Exactly, There is nothing Novel here, even a middle school Chatgpt user would have known this.