Regarding swapping, are you sure it's not just from ram backed tmpfs partition? For me that was it and most of the swap was from /tmp. So I created an agressive job that runs frequently to delete all tmp files not used in the last ~2 hours without any command holding handles to it.
The other thing I found was that there were a bunch of leftover processes from prior coding agents forking commands and just leaving them. So I had to build a system into my agent spawner to automatically launch each agent session in a cgroup whose processes all get killed on task completion.
Deduplication in ZFS needs 5G of RAM for 1TB of logical data - if you expect your agents to produce 2TB of data, you should provide 10G of RAM for deduplication.
[1] https://stackoverflow.com/questions/75452073/understanding-e...
Builds of some of projects I've been working on required 20Gbytes and more of disk space.
What are realistic scenarios in which you change a byte back?
Copy on write worktrees do absolutely nothing to solve this, either. The build artifacts alone will do it
It's all a bit adhoc - agents still need to pull (or I need to keep the main enlistment up to date on these machines) - I'm thinking of using containers or something else like github actions keep things cleaner, but right now it's "good enough".
Regardless I'm pretty happy with how it's working so far. Previously I'd be letting agents do work in the background and they'd kick off builds in the middle of a meeting or something and I'd start dropping video streams until I killed everything.
So, only first worktree is built: - less space needed. - less time spent on build, which is usually caused by switching branches. - you are not required to merge/rebase of master branch to solve conflicts, which also cause long time rebuilds.
Switching to Orbstack solved the problem for me because of their memory management efforts. Just wanted to mention it in case it leads to an easy fix for someone else too.
- my approach to using git worktrees for all work. Not entirely surprising, the duplicated build artifacts in each worktree being the main culprit here. There were some limited gains to be had by just being more efficient with how builds were managed but ultimately I just had to limit the number of concurrent features I was working on in a couple of particularly large repos.
- more insidious was the fact I was actually hitting memory limits more than I realised and causing my mac to have to swap heavily. The system was surprisingly still quite performant while this was happening... until it wasn't. Unfortunately as best I can tell the only "fix" when this happens is to restart the machine. There's no way to have the system reclaim any of the swap space once it's written to disk. Which if you have it thrash a couple of times a day (especially easy if you leave something running overnight) adds up very quickly.