Maybe one should just search for advice from the last 20 years on how to make a human development team more effective, and do that stuff.
It’s funny how this advice has always been around, but we needed to invent this new kind of idiot savant developer to get the human developers to want to do it…
Am I the only one who worries about agents creating malicious/unsafe code to execute?
You can think of it literally as turning prompts that are non-deterministic into code. But it is also the idea of documenting all the things you learned while doing something. It’s what companies do when they invent process for humans (eg. “expense reports must be approved by your manager and finance before you get reimbursed”)
AI is a great to for exploring and experimenting with ways to do things- but once you know how you want it done, AI is just a liability because it might randomly decide to do things some other way, and it might do the thing differently tomorrow even though that’s not what you want.
For instance, this skill of web development: https://raw.githubusercontent.com/vercel-labs/web-interface-...
That’s too much for a model to carry in its context while it’s trying to do actual work.
Far better is to give that skill.md to a model and have it produce several hundred lines of code with a shebang at the top. Now you haven’t got a skill, you’ve got a script. And it’s a script the model can run any time to check its work, without knowing what the script does, how, or why - it just sees the errors. Now all your principles of web dev can be checked across your codebase in a few hundred milliseconds while burning zero tokens.
TDD is codification too: codifying in executable form the precise way you want your logic to work. Enforce a 10ms timeout on every unit test and as a side effect your model won’t be able to introduce I/O or anything else that prevents parallel, randomized execution of your test suite. It’s awesome to be able to run ALL the tests hundreds of times per day.
Constantly checking your UI matches your design system? Have the model write a script that looks at your frontend codebase and refuses to let the model commit anything that doesn’t match the design system.
Codification is an insanely powerful thing to build into your mindset.