MemGPT – LLMs with self-editing memory for unbounded context

Re:MemGPT – LLMs with self-editing memory for unbounded context by pacjam on Monday October 16, 2023 @02:32PM (#37904084)

Recursive summarization is a simple and popular way to provide the illusion of infinite context (when you need to free up space, just summarize the oldest N messages into 1 summary message). It's lossy and you'll inevitably lose important information, but it should degrade relatively gracefully. In MemGPT we use (implicit) recursive summarization on top of all the explicit memory management.

[ Parent | on HN ]

Re:MemGPT – LLMs with self-editing memory for unbounded context by ASalazarMX on Monday October 16, 2023 @02:56PM (#37904457)

Would this be the same method used to assign a title to your chat based on the first prompt? It's surprisingly effective at getting the core idea most of the time.

[ Parent | on HN ]

Re:MemGPT – LLMs with self-editing memory for unbounded context by pacjam on Monday October 16, 2023 @03:09PM (#37904680)

Thanks for your interest! Question - does the title of the chat ever change after it's first assigned? If so, using a recursive summary to refresh the title sounds like a reasonable idea (especially if you're already computing a summary to extend context).

From what I remember the title in ChatGPT gets set once after a few messages, in which case I'd assume it's generated with a special "title generation" prompt (that gets the first few messages as input).

In either case since I don't work at OpenAI I can't tell you for sure ;)

[ Parent | on HN ]

Re:MemGPT – LLMs with self-editing memory for unbounded context by icelancer on Monday October 16, 2023 @04:47PM (#37906366)

This is how we do things at our work with the API and chunking since we don't have the 32k API. It works fairly well in limited windows.

[ Parent | on HN ]