e1g

+1; using Sol 6.1 in Ultrafast is noticeably faster than DS 4.1 Flash in real world tasks where marginal intelligence and token efficiency matters. Yes it costs more, but that’s a function of how valuable is the thing you are iterating on.

I'm doing some testing, and while my writes are <1s, I'm seeing E2E delivery latency of p95=2.5s / p99=7.5s. Is this roughly expected?

A hallmark of good architecture is adaptability to unexpected changes in requirements. Planning ahead helps with 'known unknowns', but it's impractical when building across N years in a dynamic environment - "knowing ahead of time" is just not possible for anything non-trivial. You need strong architectural primitives that don't scale based on developers' omniscience.

For example, say you have a workflow where, when someone signs up, you generate a user label, e.g., `$firstName $lastName`. You decide to move that to a function that might consider their personal title, preferred name, etc. Currently, it's a pure sync operation. Then, you discover people don't fill out that form at all, but some log in with Google, and you can use the name there as a fallback. Under flexible, this change is local: you can put that into your `createUserLabel(userInfo)`, and it can decide to fire off an API call to fill in any missing data, etc. In Promises land, this would taint the entire tree of everything everywhere that called that method, and all of those things must evolve or be refactored. In an Effectful system, this (previously unplanned) change remains isolated to that one function. Multiply that by every decision over multiple years for software looking for PMF, and requiring developers to "know ahead" severely slows down your ability to evolve.

This problem is commonly known as “colored functions”[1] and the only option in JS is to have some other runtime coordinate your function execution, and in JS that solution is Effect[2]

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

[2] https://effect.website/

“safehouse <harness>” hides and protects your filesystem from the agent CLI - it cannot read/write/delete anything outside of the current folder, no matter what Bash/code it tries (short of finding some 0-day within MacOS sandboxing). We added exceptions for dev tooling (so eg NPM works), but generally keep same defaults (eg no Docker access by default as having that socket bypasses all protection)

On a Mac, you can do this via OS-native Seatbelt sandboxing - you define a static text file and it locks down the process so it can't read/write anything it's not supposed to. The syntax is a bit archaic, so I built a Bash wrapper around it to dynamically detect which agent you use and from what dir, and then generate the policy to lock down its process - https://github.com/eugene1g/agent-safehouse

It took me 3 months to go from “I will never use it” to “I will never not use it”. Plus it’s always fun to watch the creators of Effect get confused about which brackets to where. The syntax is archaic, luckily we don’t write syntax by hand any more.

Unfortunately, no, MacOS sandboxing does not allow that type of blocking. Only broad categories like "local" vs "remote", but no selective targeting based on domain names or IPs.

Safehouse author here - glad you found it helpful. It didn't even occur to me to add --offline mode because my mental model is anchored in agents which often require network access. Until we add this, the easiest option is to create a custom policy and then reference it with Safehouse (I usually setup shell aliases/funciton for this)

  ;; in ~/.config/agent-safehouse/no-network.sb
  (deny network*)

  safehouse --append-profile="$HOME/.config/agent-safehouse/no-network.sb" -- <command>

In JS land, this problem (streaming, resuming, recovering, multi-client, etc) has been fully solved by https://durablestreams.com - and it can be self-hosted, or managed via Cloudflare DO.

We actually had to revert our rollout of CF Workers because enough of our users were in Spain and couldn’t access endpoints at seemingly arbitrary times (due to the matches)

For jailing local agents on a Mac, I made Agent Safehouse - it works for any agent and has many sane default for developers https://agent-safehouse.dev

Yes, you're correct about 'no nested sandboxing'.

This also applies to sandboxing an Electron app: Electron has its own built-in sandboxing via sandbox-exec, so if you're wrapping an Electron app in your own sandboxing, you have to disable that inner sandbox (with Electron's --no-sandbox or ELECTRON_DISABLE_SANDBOX=1). In the repo, I have examples for minimal sandbox-exec rules required to run Claude Code[1] and VSCode[2] (so you can do --dangerously-skip-permission in their destop app and VSCode extension)

[1] https://github.com/eugene1g/agent-safehouse/blob/a7377924efa...

[2] https://github.com/eugene1g/agent-safehouse/blob/a7377924efa...

Claude Code and Codex quickly figure out they are inside sandbox-exec environment. Maybe because they know it internally. Other agents often realize they are being blocked, and I haven't seen them go haywire yet.

Big love for Pi - it was the first integration I added to Safehouse. I wanted something that offers strong guarantees across all agents (I test and write them nonstop), has no dependencies (e.g., the Node runtime), and is easy to customize, so I didn't use the Anthropic sandbox-runtime.

SBPL is great for filesystem controls and I haven’t hit roadblocks yet. I wish it offered more controls of outbound network requests (ie filtering by domain), but I understand why not.

Yes, Safehouse should work for xcodebuild workloads in the way you described - try to run it, watch for failures, extend the profile, try again. Your agent can do this in a loop by itself - just feed it the repo as there are many integrations that are not enabled by default that will help it.

Thanks, Codex helped to put that together in like 20 minutes. Try feeding your agent the idea about an interactive config builder, give it the upstream URL with your condos, and see if it can whip up something for you.

That’s a great idea. I think I’ll restructure the entire project to be based around a collection of community managed rules, a UI generator to build a custom text file from those rules, and an LLM skill so people can evolve their policies themselves. The Bash script will remain in the background as one implementation, but shouldn’t be the only way.

Let’s make something so popular and useful that they can’t drop it.

IYKYK, it’s the new Bootstrap!

The alternative would be “no site”, which is still somehow worse.

Yes, for about a decade. But it’s available everywhere, and still works - and protects us - like brand new!

Fair! You don’t actually need to install anything and can just generate a text file with the security profile for sandbox-exec. You can do that online at https://agent-safehouse.dev/policy-builder.html

Alternatively, you can feed these instructions to your LLM and have it generate you a minimal policy file and a shell wrapper https://agent-safehouse.dev/llm-instructions.txt

Testing in prod! Thank you, just fixed that typo.

playwright-cli works out of the box, and I just merged support for agent-browser. If you end up testing out Safehouse, and have any issues, just create an issue on GitHub, and I'll check it out. Browser usage is definitely among my use cases.

Thank you for your work - I have sent many of your links to my people.

Your point is totally fair for evaluating security tooling. A few notes -

1. I implemented this in Bash to avoid having an opaque binary in the way.

2. All sandbox-exec profiles are split up into individual files by specific agent/integration, and are easily auditable (https://github.com/eugene1g/agent-safehouse/tree/main/profil...)

3. There are E2E tests validating sandboxing behavior under real agents

4. You don't even need the Safehouse Bash wrapper, and can use the Policy Builder to generate a static policy file with minimal permissions that you can feed to sandbox-exec directly (https://agent-safehouse.dev/policy-builder). Or feed the repo to your LLMs and have them write your own policy from the many examples.

5. This whole repo should be a StrongDM-style readme to copy&paste to your clanker. I might just do that "refactor", but for now added LLM instructions to create your own sandbox-exec profiles https://agent-safehouse.dev/llm-instructions.txt

> An agent inside sandbox-exec still has your AWS keys, GitHub token, whatever's in the environment.

That's not the case with Agent Safehouse - you can give your agent access to select ~/.dotfiles and env, but by default it gets nothing (outside of CWD)

Love the feedback -

For handling global rules (like ~/.gitconfig and ~/.gitignore), I keep a local policy file that whitelists my "shared globals" paths, and I tell Safehouse to include that policy by default. I just updated the README with an example that might be useful[1]. I also enabled access to ~/.gitignore by default as it's a common enough default.

For process management, there is a blurry line about how much to allow without undermining the sandboxing concept. I just added new integrations[2] to allow more process control and lldb, but I don't know this area well. You can try cloning the repo, asking your agents to tweak the rules in the repo until your use-case works, and send a PR - I'll merge it!

Alternatively, using the "custom policy" feature above, you can selectively grant broad access to your tools (you can use log monitoring to see rejections, and then add more permisions into the policy file)

[1] https://github.com/eugene1g/agent-safehouse?tab=readme-ov-fi...

[2] https://github.com/eugene1g/agent-safehouse/pull/7

Claude: can escape its sandbox (there are GitHub issues about this) and, when sandboxed, still has full read access to everything on your machine (SSH keys, API keys, files, etc.)

Codex: IIRC, only shell commands are sandboxed; the actual agent runtime is not.

No doubt they are getting better, but even a 0.1% chance of “rm -rf” makes it a question of “when” not “if”. And we sure spin that roulette a lot these days. Safehouse makes that 0%, which is categorically different.

Also, I don’t want it to be even theoretically possible for some file in node_modules to inject instructions to send my dotfiles to China.

Correct, this is for skipping permissions (safely), but does nothing for skipping questions.

Pure TUI is solid - I’ve been running all my pets inside that cage for several weeks with no issues. Auto-updates work, session renewals work, config updates work etc.

But lately I’ve been using agents to test via browsers, and starting headless browsers from the agent is flakey. I’m working on that but it’s hard to find a secure default to run Chrome.

In the repo, I have policies for running the Claude desktop app and VSCode inside the same sandbox (so can do yolo mode there too), so there is hope for sandboxing headless Chrome as well.