LLMs are designed to circumvent our ability to tell who knows what they're talking about. Our fundamental intuition about computer programs - garbage in, garbage out - has been broken. Now everyone can sound the same. But those who do know what they're talking about understand it's just a tool, those who don't think they're like Neo and just learnt kung fu.
I recently reviewed a large PR from a decent developer. I wrote some of the requirements after meeting with stake holders and did my own investigation on potential solutions.
The PR was full of bugs, missed half the requirements and the code that was there wasn’t even fully tested.
This person has successfully delivered complex things before. So what the hell? And this is not the first time I’ve thought their usage of agents and conclusions from ai assisted explorations were not right or could be better.
There’s still some weird human element. Though I’m noticing that responses are getting better with less “prompt engineering” so maybe this will all be gone soon.
Was this a developer who has been forced or pressured to use LLMs?
I know that should that time come for me, I'm not going to burn myself out wrestling to make something good.
My experience has been good, if I get the project off on a good path, with clear principles, patterns, and practices that I want to see enforced.
"List every file you need, read them in one pass, make all edits, run tests once." It stopped the read,edit,read,edit loop.
The second was a stopping rule I added : "After two failed attempts at one symptom, stop editing and get a diagnosis first"
Different tasks, different tools, different prompts, or different ability to evaluate the results, likely some combination of all of the above.
- Tools: I can't get the ChatGPT web interface, with any prompts, any models, any amount of retries, with any combination of settings, to respond to a coding task with anything other than <snippet ignoring multiple explicit requirements for what should be a 20 SLOC task> <here's how you make it actually prod-ready (often stating that it just refused to implement something I asked for)> <paternalistic essay about how I shouldn't use my vibe-coded URL parser to send a man to the moon>. The underlying models are a fair bit better. Fully controlling the system prompt helps a lot. Agentic flows paper over a lot of those deficiencies. Anthropic models (and web interfaces) work a lot better for codegen for me out of the box. Agentic flows make that delineation much sharper.
- Tasks: I don't like the concurrent data structures LLMs write. Its opinions about the novel pseudo-probabilistic algorithms I care about are wrong. The level of hand-holding I need to get proper matrix codegen is insane. It works extremely poorly when operating at a meta-type level and trying to design a cohesive API. But, if you're not doing those things, who cares? I vibe-slopped a new programming language yesterday just to try it out with a new harness I previously vibe-slopped. At this stage, it doesn't have to be correct. If I get it kind of working just a little and think the results look promising then I can do better later if I want. A lot of companies are in (or at least think they're in -- IME this mentality bites founders much earlier than they expect much of the time) that stage.
- Prompts: LLMs are much more sensitive than you might expect to minor variations in prompts. The classic example I hope everyone knows by now: There are precious few scenarios where your interaction with an LLM should be a very long session repeatedly telling it the mistakes it's made. Almost anything else is better, up to and including (as a bad solution but still better for codegen output than the baseline) still having a very long session but asking the agent to rewrite the previous messages as if it had succeeded and then having the person manually edit the conversation.
- Evaluation Ability: Really? You thought +3k SLOC was the best way to solve that feature? I have a whole SOTA event loop and multi-processing framework in 3k SLOC, and when interacting with most companies' codebases you really ought to be deleting code with the average feature you implement, not adding code. Like you said, the code smells are everywhere. Like, slop is fine from time to time (easily 30% of the code I write for work is slop, largely because it serves one-off tasks), but some of the people pushing it just can't tell the difference. Just a month ago, somebody (outside of $WORK) was showing off a fancy new UI feature on some geo-rich project. It had a small bug. The implementor said they knew about it. The problem is that "if you know you know," and the only reasonable way the code for that kind of feature should have that kind of bug is if it has major problems which just aren't surfacing much in this particular demo. I suggested a couple alternative test cases to optimize toward, and a week and a couple rewrites later it was in a much better place. There were tens of thousands of tests before I made that suggestion, dozens of times the LLM confirmed the system was working as intended, and on and on and on. None of that matters if you need to build something that works and know that it works. The tests were all shit, as were the false assurances.
If you're looking to make it better, there are some tweaks you can make to your CLAUDE.md or what have you to help a bit. I have a lot that I do to make mine generate passable code without requiring constant human intervention, but something it responds well to is (1) pinning down what you care about re abstractions, and (2) giving it explicit permission/guidance to spawn sub-agents to make that happen. A lot of the problems with LLM-generated code are fixed if you fix the abstraction issue, and many of those fixes are relatively "mechanical" checks you can apply:
1. When reading the name of an abstraction, do you have a good idea of what it does?
2. What about the name and signature?
3. Does the implementation match that intuition?
4. Does any part of the codebase reach past the intended abstraction?
5. Does your code require nonlocal reasoning? (side-note: this is very important for humans, but it's more important for models since their harnesses aggressively try to avoid bloating context windows, so it's quite common that if you need two lines of code far apart in a file to relate to one another then you'll see downstream bugs)
6. Do all of the abstractions pull their weight? Are they all even used?
7. Is it possible to delete 80% of the suggested code and have an equivalently working product?
8. Do existing abstractions/conventions solve the problem you wrote a new abstraction for?
And so on. You obviously can't compress all good engineering judgement into a brief model context window, but you can get a lot more than you might expect. Whatever it is that you see agents doing poorly, try to come up with a general principle you think it should try to respect, and write that down for your future projects.
This slope is hardly any different had they gotten an offshoring team at the lowest bid.
The only jobs that were created were at their locations, at the expense of local teams.
Because it’s an infinite loop in essence. There are always things to improve. There are always things to fix. There are always bugs in software. There are always more cases to check. There’re always better ways to do things.
You can add new code indefinitely with diminishing returns. Each addition introduces new bugs. So after a point it may become negative returns.
As human engineers we make tradeoffs. We prefer simple and clean over more complete but bloated.
Without a stop condition Claude can “improve” the code indefinitely.
So when a LLM writes new software and the LLM sweeps keep finding a ton of issues. Yea, but the LLM wrote that code in a few hours, what may have been weeks of work. The sweeps its running is finding things, that may have taken years to remove as customers report issues.
Combined with LLMs tend to "over find" issue things that we will never bother with as human written code. We may say "too trivial to fix, not a issue", ... bla bla but for a LLM its a issue to fix, so it looks like LLM code has more issues.
And while we may pat ourselves on our back for finding X bug that created data loss or a crash, there is probably a dozen high to medium bugs hidden somewhere to be found by the customers (yikes!). Where as LLMs then to reduce that vector after a sweep to barely a few medium or low items.
This is what i always need to tell people. LLMs are not just doing the work of humans at a 10x faster speed, they are also solving bugs and other issues what may have taken years for humans to iron out, what a LLM does in maybe a 24h bug hunt session.
it’s all very much over-eager junior level.
You have not seen the over smart senior dev that tries to one-liner everything. Or that makes code so complex, that its a "job secure" feature because nobody else can figure out what the ** is going on.
I noticed that there is this perception that medium or senior devs are the panicle of coding, and good practices when most are not. A lot of are simply restrained because companies force specific coding standards and left to their own devices, they will love to run amuck. Remove those handrails and your going to see code that makes most LLMs blush. lol
after some thought it will give me 15 to 30 suggestions of what needs fixing.
That simply means that your collogues never bothered to run a bug hunt sweep after their work. If they did so, just like a human dev writing something and having the work checked by another dev, this will have been reduced a minimum. People seem to have this thinking that LLMs need to write perfect code from the start, when its trained on our human code, with all its issues. Just like if we wrote code manually, and then somebody checks the code before pushing and committing it, do the same for LLM code.
!!! LLM do not mean that good coding and deployment practices are suddenly useless !!!
You have to carefully manage the project scope, and catch the ai getting stuck. There is a stars aligning point where you are matching the model to its scope on a clearly defined goal to iterate towards and its mind-blowing. And then if you go even a bit over that point it's perf falls off a cliff.
And it changes with each model with each new release.
For example, I wasted a bunch of usage today giving Fable 5.2 8 sub-orch opus high agents that each were given bunch of chatgpt subagents. It was fun watching all the churn, and I'm trying to find some way to use up all my openai usage.
But in the end I made much less progress/made a bunch more slop code than if I had just narrowed project scope and gave fable a smaller easier task with less agents under it. And now my weekly usage is gone in 2 days.
The changes always come in monster one-shot PRs ofc - too much for a human to read - so I just feed the branch to my Claude (identical to the one they’re working with) and after some thought it will give me 15 to 30 suggestions of what needs fixing.
I’m still not understanding why this happens?