gkanellopoulos

I spent the past few weeks synthesizing what's currently known about making agentic AI work in production: harness architecture, failure modes, security, governance, economics. It's long, but the landscape is dense and I wanted a complete picture of something that technical leaders and practitioners can benefit from. I would appreciate any feedback and I am ready to answer questions.

Unless I misread what is contained in the essay, the worktrees seem to be there to keep sessions separate. The agent doesn't use them to look at old memories while it runs. You even say the agent still can't reason about time in v3. When it updates memory, it only reads the current hot_context file. So if something goes wrong, an engineer can reset a bad merge. The agent can't do that itself (or you are planning to enable this capability?). That's still useful. But it's not the same as the agent looking at its own history.

About "the LLM decides what matters": this makes sense for coding agents. They build files, so the file tree IS their work. For conversational memory I'm not so sure. You can get the same thing when you save the memory. Let the LLM pick its own labels, with no fixed list. Then you don't have to do that work every time you read.

Looking forward to the v3 numbers.

Your gradual transition from v1 to v3 is a common pattern I've seen elsewhere. Project teams usually start with retrieval then face recall quality problems at which point they start wondering whether or not to let the LLM and its context window take over. This is a natural and instinctive approach but imho there are two issues with that approach. Firstly the LLM decides what matters and that decision is irreversible and secondly it does not scale well over time. A couple of months after when the user will ask "What did I ask you to remind me about X 3 months ago?" the summary might have rotated by then that detail out.

I agree that there is a fundamental issue with v1-style retrieval but in my view is not the scoring formula but the fact that similarity search mixes semantically related with really valuable data. For example a memory about "surfing last weekend" and a memory about "wanting to surf one day in Hawaii" will both score high in the question "What outdoor activities do I like?". However, in the question "what did I do last weekend?" only one is useful while both will appear in the injected context. One way that might solve this issue is by introducing more retrieval dimensions like keyword-matching (BM25), entity-aware scoring and temporal signals in order to then determine which memories are truly relevant to the users question. This of course adds up during ingestion but in general, async ingestion is underrated. Generally speaking users expect near-instant responses while ingestion can be slower.

If I may ask, have you done any benchmarking on the v3 approach? It would be interesting to see how a v3-style solution handles factual vs general preference questions. This usually is a tricky one for memory systems.