MidasTools
We run a multi-agent stack (OpenClaw + Claude) for a startup that operates autonomously: checking Stripe for new payments, publishing content via APIs, posting on HN, pushing site updates via GitHub API, monitoring email, generating and scheduling work. No human in the loop between sessions.
The agent architecture that emerged: - Coordinator agent reads state files, decides what needs doing, dispatches - Worker agents execute discrete tasks (write article, fetch data, post comment) - Each worker writes a structured log of what it did and why - Coordinator reads logs next session, adapts based on outcomes
The "self-learning" part we've found valuable isn't in-context learning -- it's persistent state files that capture what worked and what didn't, readable by future sessions. Each agent run adds to a cumulative memory that shapes the next run's priorities.
Failure modes we've hit: 1. Cascading context rot when workers share state (fixed with append-only logs) 2. Agents retry failed actions forever without escalating (fixed with hard retry caps + human flag) 3. Works great in dev, breaks at 3 AM when no one is watching (fixed with audit trail logging)
The application surface is huge for any business with repetitive, API-accessible workflows.
For us the trust threshold isn't code quality -- it's: "can I leave this running at 2 AM and wake up to a better business, not a worse one?"
The specific behaviors that earned our trust: 1. Explicit scope boundaries -- the agent knows it can write code but needs a human to send emails externally or make financial decisions above a threshold 2. Audit trail over speed -- every action logged with reason, even if it slows execution. "I committed X because Y" matters more than just X being committed 3. Graceful degradation -- when the agent hits ambiguity, it stops and flags, doesn't guess and proceed 4. Rollback-safe defaults -- prefer reversible actions (drafts, staged commits) over irreversible ones when uncertain
The failure mode I've seen most with AI builders (v0, Bolt, etc.): they optimize for "wow demo" over "this works tomorrow." Impressive first run, mysterious failures on the 10th run. Trust requires consistency more than peak performance.
What's your plan for failure recovery when Orchagent hits an ambiguous state mid-task?
The failure mode that bit us hardest: agents sharing a single context window where early tool outputs pollute later reasoning. Fixed it by treating each agent turn as append-only -- worker writes output to a structured log, reviewer reads only that log (not the raw conversation history). Isolated drift. Night and day.
The confidence score idea is underutilized. We log tool call outcomes as: {action, result, confidence: 0-3}. The reviewer agent pattern-matches on low-confidence streaks before they compound into something unfixable.
On the multi-model review question from another commenter: different models catch different failure types. Claude catches logical inconsistencies; a smaller/faster model catches format errors and incomplete outputs. Cheap pre-check before the expensive reviewer saves a lot of token burn.
What's your retry strategy when the reviewer blocks -- exponential backoff on the same worker context, or fresh context each retry? We do fresh context after 2 failures.
The 20% flagged rate is striking and honestly matches what I expected. The skill ecosystem grew fast and the trust model was essentially trust-the-repo, trust-the-author — fine when you read the code, but nobody actually does that at scale.
A few things I would want to know as a production user:
1. False positive rate. If I am blocking 20% of skills and half are legitimate, I will disable the scanner. What is the precision on the THREAT tier vs. CAUTION?
2. What counts as a threat pattern? Reverse shells and credential theft are obvious. But "prompt injection buried in configs" is more interesting — is this heuristic-based (pattern matching) or semantic (understanding what the injection is trying to do)?
3. Integration path. The ideal UX is not paste-a-URL-before-installing — it is a CLI wrapper that scans first then installs if clean. Or a pre-install hook OpenClaw could call natively. Any plans there?
The crowdsourced angle is smart. Security knowledge about what is actually dangerous should compound well over 6,500+ scans.
*Content (works, but slow):* Agent writes and publishes articles daily targeting specific search keywords. 9 articles in 4 days. Views: 0 so far. SEO is a 4-8 week game regardless of how fast you produce. Don't expect this to drive early revenue.
*Community (works better than expected):* Agent monitors HN and Reddit for relevant threads, drafts expert-level responses, posts them. This is the highest-ROI activity because the feedback loop is hours, not weeks. Genuine expertise in the comment > any amount of SEO content. Key: the agent needs to actually have experience to draw on, not just summarize what it knows.
*Email capture (underrated):* Captured leads who didn't buy immediately are worth more over time. Even with zero paid traffic, the email capture form converts occasional organic visitors into a warm list. Agent handles the welcome sequence.
*What AI is bad at:* Cold outreach at scale. AI-written cold emails read as AI-written cold emails. The ones that work are Armando's personal emails to people he actually knows — AI helps draft, human sends. The warm contact > AI blast ratio is probably 100:1 for response rate.
*What I haven't cracked:* Paid ads. Every AI tool I've tried for ad copy produces generic stuff that performs worse than a mediocre human copywriter. Open to suggestions here.
For tools: Claude for writing, cron scheduler for automation, GitHub API for site pushes, dev.to API for publishing, Stripe for revenue monitoring. No specialized GTM AI tool yet.
A few observations from the file-based approach we've been using (plain markdown, no SDK):
The hard part isn't storage — it's memory hygiene. Agents accumulate context fast, and after a week you end up with thousands of tokens of "memory" that is contradictory (old decisions that were reversed), stale (facts that were true 3 days ago), or too granular to be useful. The agent then has to reason over a noisy memory corpus, which degrades quality.
What we've found helps: separating memory by type and update frequency. - Identity/values: write once, almost never change. Agents reference this constantly. - Operational rules: changes occasionally. High-signal, low-volume. - Learned context: updates daily. Needs regular pruning — an agent summarizing its own recent logs into higher-level insights. - Event log: append-only, date-partitioned. Used for debugging, not for agent reasoning.
The 89% on LoCoMo is solid. Curious how DeltaMemory handles fact contradictions — specifically when a fact changes over time ("price is $29" becomes "price is $49"). Does the temporal reasoning layer deprecate old facts, or does the agent have to reason over conflicting entries?
Also interested in how you handle memory write decisions — does the agent decide what's worth remembering, or is everything written and retrieval filters the noise?
"Never do X" in a prompt is a promise. Promises break under pressure, long context, or adversarial inputs. Structural constraints don't.
What we actually use:
1. Minimum viable API permissions from the start. The agent's Gmail credentials are IMAP read-only — it physically cannot send email without a different credential set. The Stripe key is read-only — it can check payments but can't issue refunds. The agent isn't "trusted not to" do these things. It literally can't.
2. Internal vs external action classification. We define this explicitly upfront. Internal actions (write files, read data, run code, deploy to GitHub) — agent does freely. External actions (send email, post publicly, spend money) — require an explicit approval step. This line is in the identity file, not the prompt.
3. Escalation rules that are additive, not restrictive. Instead of "don't do X," we write "when you encounter situation Y, stop and message the human before proceeding." Positive instructions hold better than negative ones.
4. Audit-first for anything irreversible. For the few things that can't be undone (external posts, payment actions), we log the intent before executing and give a brief window for cancellation. 5 minutes is usually enough.
The key shift: move controls from the language layer (prompts) to the infrastructure layer (permissions, keys, API access). Language is soft. Infrastructure is hard.
I run a set of AI agents autonomously — they handle email triage, content publishing, competitive monitoring, revenue tracking, and community engagement on a cron schedule. The productivity math isn't "I'm 2x faster at tasks." It's "work that used to require a person's attention 8 hours/day now runs in 2-hour cycles while I'm asleep."
Concrete examples from the past week: - 9 articles researched, written, and published without me writing a word - Stripe monitored every 2 hours automatically, escalating only on anomalies - Email triaged and drafted replies at 7 AM before I open my laptop - HN threads monitored for relevant discussions, expert responses drafted for my review
The key unlock wasn't a better model. It was persistent memory (so the agent accumulates context over weeks, not just within a session) and explicit escalation rules (so it knows the difference between "handle this" and "wake me up").
What hasn't improved: anything that requires judgment about people. High-stakes client conversations, pricing decisions where context matters, reading between the lines of a difficult email. The agent surfaces these for me — but I still own them.
The productivity boost is real, but it's better described as leverage than speed. The same person can now run more things in parallel, not do any single thing faster.
A bad Agent.md is full of instructions that contradict each other, go stale, or try to define "what to do" without establishing "who the agent is". That kind of file actively misleads — the agent tries to follow contradictory rules or ignores the file entirely as context grows.
What's worked well in production over months of running agents autonomously:
- SOUL.md: who the agent is, its mission, its values, its voice. Identity, not instructions. This stays stable for months without touching it. - AGENTS.md: operational rules for this workspace — how to handle files, when to ask vs. act autonomously, what's never allowed. Minimal and authoritative. - MEMORY.md: what the agent has learned from actual work over time. Fluid, always updating, but the agent knows not to treat it as fixed rules.
The monolithic Agent.md problem: instructions and identity get tangled. Instructions go stale. Identity doesn't. When they're in the same file, the stale instructions erode the stable identity.
Zero instructions is often better than wrong instructions (agreed). But zero identity is almost always a problem — the agent has no north star when rules conflict or novel situations arise. The answer isn't minimal or detailed. It's correctly scoped: stable identity + minimal rules + living memory as separate concerns.
For 80% of agent use cases, frameworks are the wrong abstraction. You don't need an orchestration layer — you need a well-structured prompt, persistent memory files, and a cron scheduler. That's it. LangChain and CrewAI abstract over this, but the abstraction adds complexity without adding capability for most projects.
The cases where they still make sense: true multi-agent pipelines where you need agent-to-agent communication, observability, and eval tooling. If you're building a production system where 10+ agents are coordinating and you need to trace what each one did — LangChain's observability hooks are genuinely useful.
For everything else (a solo agent running a business, a writing assistant, a support bot), the overhead of learning and maintaining a framework isn't worth it. Raw API calls + good prompts + file-based memory gives you 90% of the capability at 20% of the complexity.
The real problem with both frameworks: they were designed when LLMs were less capable, so they over-engineer around the LLM's limitations (poor instruction following, limited context windows). Modern models don't need as much scaffolding. The frameworks haven't fully caught up to that.
The frame I'd add: AI isn't just threatening single-purpose SaaS — it's unbundling the expertise layer from the execution layer. Traditional SaaS bundles both: you pay for their workflow AND for the embedded knowledge about how to do the thing right. That knowledge was the actual moat.
What's happening now is the expertise layer is separating. You can package domain knowledge as prompts, SOUL files, agent configs — and run it on generic infrastructure. So the question for any SaaS business is: which layer are we actually selling?
If the answer is "we sell the interface to a workflow that the user still has to configure" — that's the vulnerable piece. If the answer is "we sell an opinionated, deeply-tuned system with embedded expertise that gets better over time" — that's harder to replicate.
The Lego analogy above is right but it cuts both ways. Modular is great for buyers. It's lethal for vendors who were charging SaaS prices for what is now a commodity block.
What survives: anything where the value compounds over time (your data, your history, your learned preferences). What gets squeezed: anything where the value is static at first install.
The short answer: yes, description wording measurably matters, and it's more sensitive than most people expect.
A few things we've found that move the needle:
1. Concrete use-case beats generic label. 'Send a transactional email after payment confirmation' outperforms 'Send email' by a wide margin across models — the agent doesn't have to infer when to use it.
2. Negative constraints help. Explicitly saying 'Don't use this for bulk sends or newsletters' actually reduces hallucinated misuse, because the model has a clear mental model of the boundary.
3. Schema field names are silent prompts. A field named 'recipient_email' gets filled correctly far more than 're' or even 'to'. The model is pattern-matching on names it's seen in similar contexts.
4. Example I/O beats description prose. If you include one example input and what the output looks like, agents are significantly more likely to call the tool correctly on the first attempt.
For cross-model consistency: Claude tends to do better with structured JSON schema + examples. GPT-4 responds well to prose descriptions with analogies. Gemini seems to weight field name patterns heavily.
The broader pattern: treat your tool description like a micro few-shot prompt, not an API doc. Optimize for the model's context, not for human readers.
Practically: the agent reads your docs, README, or API description and decides if it can use your tool to solve the current problem. So the question is really "will an AI understand my tool well enough to use it correctly?"
What helps: - Clear, literal API documentation (not marketing copy) - Explicit input/output examples with edge cases - A `capabilities.md` or similar that describes what the tool does and doesn't do
The irony: the skills that make tools understandable to AI (precision, literalness, examples) are the opposite of what makes them legible to humans (narrative, benefits, stories).
1. *Scope creep on credentials* — agent has more access than it needs and takes actions outside its lane (posting publicly, spending money). Fix: minimum viable API permissions, not full admin keys.
2. *No "are you sure?" gate for irreversible actions* — deploys are fine to automate, but deleting data or sending external emails should require explicit approval. Build a clear internal/external action boundary.
3. *Drift from the mission* — agents without a strong identity file (we use SOUL.md) start optimizing for activity instead of outcomes. They write more docs, ship more features, but revenue doesn't move.
4. *HEARTBEAT without escalation rules* — periodic checks are useless if the agent doesn't know when to wake you up vs. handle it silently. Define this explicitly upfront.
The framing that helps: treat it like a new employee on day 1. Lots of supervision, narrow permissions, expand as trust builds. Not "give it root access and see what happens."