AI Agents: Less Capability, More Reliability, Please

Re:AI Agents: Less Capability, More Reliability, Please by cryptoz on Monday March 31, 2025 @11:03AM (#43535849)

This is refreshing to read. I, like everyone apparently, am working on my own coding agent [1]. And I suppose it's not that capable yet. But it sure is getting more reliable. I have it only modify 1 file at a time. It generates tickets for itself to complete - but never enough tickets to really get all the work done. The tickets it does generate, however, it often can complete (at least, in simple cases haha). The file modification is done through parsing ASTs and modifying those, so the AI doesn't go off and do all kinds of things to your whole codebase.

And I'm so sick of everything trying for 100% automation and failing. There's a place for the human in the loop, in quickly identifying bugs the AI doesn't have the context for, or large-scale vision, or security or product-focused mindset, etc.

It's going to be AI and humans collaborating. The solutions that figure that out the best are going to win IMO. AI won't be doing everything and humans won't be doing it all either. The tools with the best human-AI collaboration are where it's at.

[1] https://codeplusequalsai.com

[ Parent | on HN ]

Re:AI Agents: Less Capability, More Reliability, Please by helltone on Monday March 31, 2025 @11:19AM (#43536048)

How do you modify ASTs?

[ Parent | on HN ]

Re:AI Agents: Less Capability, More Reliability, Please by cryptoz on Monday March 31, 2025 @11:22AM (#43536084)

I support HTML, JS, Python and CSS. For HTML, (not technically an AST), I give the LLM the original-file HTML source, and then I instruct it to write python code that uses BeautifulSoup to modify the HTML. Then I get the string back from python of the full HTML file, modified according to the user prompt.

For python changes I use ast and astor packages, for JS I use esprima/escodegen/estraverse, and for CSS I use postcss. The process is the same for each one: I give the original input souce file, and I instruct the LLM to parse the file into AST form and then write code that modifies that AST.

I blogged about it here if you want more details! https://codeplusequalsai.com/static/blog/prompting_llms_to_m...

[ Parent | on HN ]

Re:AI Agents: Less Capability, More Reliability, Please by skydhash on Monday March 31, 2025 @01:07PM (#43537265)

I took a look at your project and while it's nice (technically), for the actual use case shown, I can't see the value over something like the old Dreamweaver with a bit of training.

I still think like prompting is still the wrong interface for programming systems. Even though they're restricted, configurations forms, visual programming with nodes, and small scripts attached to objects on a platform is way more reliable and useful.

[ Parent | on HN ]

Re:AI Agents: Less Capability, More Reliability, Please by cryptoz on Monday March 31, 2025 @02:46PM (#43538323)

Appreciate you having a look and for that feedback, thanks - I do agree I have work to do to prove that my idea is better than alternatives. We'll see...

[ Parent | on HN ]