strong agree. I always have the LLM put an actual markdown doc in a docs/plans/ folder before starting work. I often, but not always review it.

Aside: it also helps for code review! Review bots can point out the diff between plan and implementation.

Some examples for the curious: https://github.com/sociotechnica-org/symphony-ts/tree/main/d...

The plan-vs-implementation diff for code review is a neat idea I had not considered. That essentially gives reviewers a spec to review against instead of just reading code in isolation and guessing at intent. Curious if you have ever had the agent update the plan doc mid-implementation when it discovers the original approach will not work, or do you treat the plan as immutable once written?

It's one of the things that surprised me when I first started using the compound engineering plugin.

I've been considering adding a review gate with a reviewing model solely tasked with identifying gaps between the plan and the implementation.

> file-based state that persists between agent invocations

Can you expand on this with a practical example?

Sure. Instead of relying on the LLM context window to remember what it already tried or decided, you write key state to disk. Something like a STATE.md that tracks "already checked X, decided Y, next step Z." Each time the agent spins up or resumes after a long tool call, it reads that file first. You can also use structured formats like a JSON task queue where each entry has a status field. The agent pops the next pending task, does it, marks it done. Cheap and simple but it completely eliminates the "wait, didnt I already do this" problem that burns tokens on long runs.

thanks

One example: I let the agent culminate the essence of all previous discussions into a spec.md file, check it for completeness, and remove all previous context before continuing.

It needs a canonical source of truth, something isolated agents can't provide easily. There are tools out there like specularis that help you do that and keep specs in sync.

...at least until we get real Test-Time Training (TTT) that encodes the state into model weights. If vast amounts of human knowledge can be compressed into ~400GB for frontier models, it's easy to imagine the same for our entire context