Wait, do you also think there are no guardrails in a production system that would fundamentally alter the security posture of a system rather than on a single laptop with a single developer would engender.
Wait, do you not believe all the AI companies aren't also trying to develop (have developed) their own sandboxing solutions?
Wait, did you even bother to think about what you were writing before you decided to write it?
Maybe the llm can carefully craft an exploit that happens when nginx reads some HTML. Maybe it found a way of hiding file system access in an import I didn’t notice.
I can completely destroy a prod service by accidentally not escaping a space in an rm command.
I’m genuinely confused by this question unless you’ve never worked on production systems in a team before. In which case that’s fine and it’s good to learn but there’s going to be a lot of material out there about deploying and safety.
What's my role here? Over the past year, it's become clear to me that there are really two distinct activities to the business of software development. The first is the articulation of a process by which an intent gets actualized into an automation. The second is the translation of that intent into instructions that a machine can follow. I'm pretty sure only the first one is actually engineering. The second is, in some sense, mechanical. It reminds me of the relationship between an architect and a draftsperson.
I have been much freer to think about engineering and objectives since handing off the coding to the machine. There was an Ars Technica article on this the other day that really nails the way I've been experiencing this: https://arstechnica.com/information-technology/2026/01/10-th...
Why do I trust the finished product if I don't trust the environment? This one feels a little more straightforward: it's for the same reason that construction workers wear hard hats in environments that will eventually be safe for children. The process of building things involves dangerous tools and exposed surfaces. I need the guardrails while I'm building, even though I'm confident in what I've built.
Good response, but more practically, when you are developing a project you allow the agent to do many things on that VM, but when you deliver code it has to actually pass tests. The agent working is not being tested live, but the code delivered is tested before use. I think tests are the core of the new agent engineering skill - if you can have good tests you automated your human in the loop work to a large degree. You can only trust a code up to the level of its testing. LGTM is just vibes.
(I am not saying one way is better than the other, it’s just different modes of engaging with risk. I obviously understand that having a helmet can and would save my life should an accident occur. The keyword here is “should/would/can” which some people believe in “shall/will/does” and prefer to live this way. Call it different faith or belief systems I guess)
IMO this pattern fails on non-trivial problems because you don't know how the intent can be actualized into automation without doing a lot of the mechanical conversion first to figure out how the intent maps to the automation. This mapping is the engineering. If you can map the intent to actualization without doing it, then this is a solved problem in engineering and be usable by non-engineers. Relating this to your simile, it is more like a developer vs. an architect where the developer uses pre-designed building vs. the architect which needs to design a new building to meet a certain set of design requirements.
Is it though? We’ve seen a lot of output at this point and it does not strike me as high…
And adding the fact that you used AI to write the supposed containment system, I'm really not seeing the safety benefits here.
The docs also seem very AI-generated (see below). What part did you yourself play in actually putting this together? How can you be sure that filtering a few specific (listed) commands will actually give any sort of safety guarantees?
https://github.com/borenstein/yolo-cage/blob/main/docs/archi...