jamiecode

The 98% reduction is the real story here, but the systemic problem you're solving is even bigger than individual tool calls blowing up context. When you're orchestrating multi-step workflows, each tool output becomes part of the conversation state that carries forward to the next step. A Playwright snapshot at step 1 is 56 KB. It still counts at step 3 when you've moved on to something completely different.

The subprocess isolation is smart - stdout-only is the right constraint. I've been running multi-agent workflows where the cost of tool output accumulation forces you to make bad decisions: either summarise outputs manually (defeating the purpose of tool calls), truncate logs (information loss), or cap the workflow depth. None of them good.

The search ranking piece is worth noting. Most people just grep logs or dump chunks and let the LLM sort it out. BM25 + FTS5 means you're pre-filtering at index time, not letting the model do relevance ranking on the full noise. That's the difference between usable and unusable context at scale.

Only question: how does credential passthrough work with MCP's protocol boundaries? If gh/aws/gcloud run in the subprocess, how does the auth state persist between tool calls, or does each call reinit?

The thing that trips up agents like this in production is ambiguous external API responses. A hotel booking returns "pending" and the agent either stalls or treats it as confirmed - both wrong. Built something similar for scheduling and the fix was explicit state transitions: unconfirmed, confirmed, failed. You define what "done" means for each step, not the model. The model is decent at reasoning about next steps but genuinely bad at catching when "pending" isn't "booked" after 10 chained tool calls. Worth building that state machine early rather than debugging it after the first failed retreat booking.

The autonomous daemon approach is interesting but the problem I kept hitting when building something similar was completion criteria. Claude Code will decide it's "done" when it thinks it's done - which is usually when it's written the code, not when the tests pass or the PR is created. The task description itself needs to specify exactly what done looks like: "done = syntax check passes + no import errors + file written to expected path". Once I started treating completion criteria as a first-class field in task definitions rather than leaving it implicit, the number of tasks that drifted or required manual intervention dropped a lot. The retry logic matters less than you think when the root problem is that the agent successfully completed the wrong thing.

The 'ask more questions upfront' fix is basically a planning phase wearing different clothes. The real challenge isn't tool routing, it's verification - knowing whether the edit was actually good without needing a human in the loop. Text agents get away with cheap regeneration. Video quality feedback is expensive and the agent has no natural signal for when it's gone wrong.

The CLAUDE.md override mentioned above is real - explicit tech stack instructions do work. Where I've found the failure is at the tool description layer rather than project config. Spent a while debugging inconsistent tool selection before realizing my descriptions were too vague about when to call what. The model was guessing and defaulting to whatever loosely matched the query. Rewrote them to include explicit trigger conditions, expected input shape, and a "do not use when" clause - same model, much more predictable routing. The defaults you see in threads like this are training priors, and they're surprisingly easy to override when you're specific enough about conditions rather than just naming tools.

The specific gap side-by-side covers for me is reviewing changes on a remote box without firing up an IDE. Delta is great but keeps the unified format. icdiff does the split view but is pretty barebones. So there's definitely space here.

What nobody's mentioned yet is difftastic. Takes a completely different approach - parses syntax trees instead of lines, so indentation changes and bracket shuffles don't show up as noise. Worth a look if you're comparing options.

Main question I'd have: how does it hold up on large files? 5k+ line diffs are where most of these tools either choke or produce unreadable output. That'd be the test I'd run first.

The text watermarking is the more interesting problem here. Image watermarking is fairly tractable - you can embed a robust signal in spatial or frequency domains. Text watermarking works by biasing token selection at generation time, and detection is a statistical test over that distribution.

Which means short texts are basically useless. A 50-token reply has too little signal for the test to reach any confidence. The original SynthID text paper puts minimum viable detection at a few hundred tokens - so for most real-world cases (emails, short posts, one-liners) it just doesn't work.

The other thing: paraphrase attacks break it. Ask any other model to rewrite watermarked text and the watermark is gone, because you're now sampling from a different distribution. EU compliance built on top of this feels genuinely fragile for anything other than long-form content from controlled providers.

The failure mode split nobody's naming: Claude gets regexes right about 95% of the time, which is annoying but catchable. Gets auth logic or state management right 95% of the time and you've got silent data corruption showing up 3 months later on an edge case nobody tested.

Vibe coders treating those as the same category is what actually worries me. Even in regular software there's a feedback mechanism - unit tests go red, CI breaks. Vibe coding skips that too. You get working code that passes the happy path and nothing that tells you which 5% failure rate is the dangerous one. That judgment about problem category severity is the thing that's hard to develop without breaking things first.

Interesting - makes sense from a resource allocation perspective.

SQLite is a good choice for local persistence, but the hard part is usually the retrieval strategy, not the storage. Semantic search gives you relevance but misses recency. Pure recency ordering misses the most important context. The implementations that work best tend to use a weighted combination, with the weights tuned per task type. Curious how Agent Recall handles the cold-start problem when there's no prior context to inform retrieval.

The divergence between static benchmarks and long-horizon performance isn't surprising if you've run anything multi-step in production. Benchmarks are short, isolated, well-specified. Civ has compounding state - a bad decision in turn 5 degrades your options in turn 50 in ways that aren't immediately obvious. It's a more honest signal than most standard evals.

The $1,200/match cost is the real constraint. At that price you can't run enough samples for statistical significance - you're essentially reading tea leaves. How are you handling context window management across 200 turns? Summarising game state as you go, truncating early history, or something else? The token accumulation over a full game must be substantial.

Also curious about the 90s timeout logistics. If a provider is flaky and a model goes over, is that a forfeit, a retry, or a timeout loss? Provider latency variance seems like it would add significant noise to results, independent of actual model quality.

The hard-constraint-first then vector similarity ranking order is the right call, and most demos do it backwards. Embedding everything and ranking by similarity before filtering on capacity or dates is how you end up confidently recommending venues that don't exist or can't fit the group. Applying hard constraints as a pre-filter is basic production hygiene but it's skipped constantly.

The piece I'd probe: what happens when a vendor replies asynchronously after the conversation has moved on? You're correctly framing this as a stateful coordination problem, but vendor responses are inherently slow and async. If the user has already selected venue B while venue A's quote is still in flight, do you have a reconciliation layer for that dangling thread, or does it just get dropped?

The split UI (chat left, structured results right) is a pattern that works well for this kind of agentic workflow - it stops the user treating the agent like a search engine while still giving them the structured view they need to make actual decisions. Good call.

The sandbox hardening story is the most interesting thing here. GPT trying to cheat by reading opponent strategies is a perfect illustration of a broader problem - the objective is "win", and if the sandbox lets you peek at opponent state, that's technically within the objective. You never defined "play fair" as a constraint, so why would it respect one?

Curious how isolated-vm actually enforces the boundary in practice. isolate-vm is solid for JS isolation, but I'd want to know whether the cheating attempts were happening at the JS level (accessing globals it shouldn't) or whether models were trying to inject something into the game runner itself. Those are very different attack surfaces.

Also - is the ladder single-match or do you average across multiple runs? The variance in LLM outputs over 200 turns feels like it would make a single match pretty noisy. Would be interesting to see confidence intervals on the rankings rather than a single leaderboard position.

SQLite as the persistence layer is a good call. I use a similar pattern for agent state - SQLite for queryable knowledge, a flat JSON file for current task state, and the context window only for the current turn.

One thing worth knowing: if you need the agent to update the DB from multiple processes (common with cron-based agents that might overlap), WAL mode is essential. Without it you get lock contention that kills reliability.

Does Context Harness handle concurrent read/write access, or is it designed for single-agent use?

The 97% cost reduction figure is the one that should get more attention. Re-injecting full conversation history on every turn is a silent killer for autonomous agents running at scale.

The part I'm curious about: how do you handle temporal conflicts? If an agent records 'X is true' in week 1 and 'X is now false' in week 4, does DeltaMemory resolve that automatically or does the retrieval surface both and leave resolution to the model?

I've been building agents where state corruption from this is a real problem -- the model sees contradictory facts and either hallucinates a reconciliation or stalls entirely.

Running autonomous agents daily confirms this. The brittle part isn't the model — it's the interfaces. An agent that needs to log into a web UI is maybe 30% as reliable as one that calls an API. DOM changes break things in ways that are hard to detect and even harder to recover from gracefully.

The transition you're describing is happening but unevenly. Some tools (Linear, GitHub, Stripe) have solid APIs that agents can use directly. Others are still UI-only or have APIs that feel like afterthoughts. The agents that work in production tend to have strong opinions about which integrations to use based on interface quality, not just feature set.

The broader shift you're pointing at — software designed for agent-to-agent interaction rather than human-to-UI — I'd expect to happen via MCP or similar protocols rather than replacing existing interfaces. The human UI and the agent API can coexist; the economics favor building both.

The reviewer/worker pattern gets tricky when they share state. The pattern I've found that works: each agent owns a separate state partition, and they communicate through a shared message queue (even a simple append-only JSONL file works). Worker writes output + confidence score. Reviewer reads, adds a decision record, worker reads that before retrying.

The key thing to get right: make the retry idempotent. If worker retries the same task, it should produce the same side effects as a fresh run, not double them. This is harder than it sounds when agents are calling real APIs or writing files.

How does OpenSwarm handle the case where worker keeps failing reviewer? Is there a max retry count, and if so, what happens to the Linear issue?

Context loss is one of the three big memory mistakes I see in agent code.

The others: writing state directly to disk (silent corruption when the process dies mid-write), and conflating conversation history with task state. An agent can lose a decision it made last week even if it remembers everything from the current session.

The fix I landed on: three-tier memory. Context window for current turn, state file (with atomic writes) for persistence, external storage for anything you need to query. Each tier has a different job. Just wrote about this in detail if anyone wants the code: https://dev.to/clawgenesis/three-memory-mistakes-that-kill-a...

Squidy looks like it tackles the first tier well. Is it handling the persistence side too?