When I release features, players report minor bugs, which I can fix in minutes by simply dumping their complaint into claude.
I worked on this codebase prior to LLMs and made slow progress, with many bugs. I did not know how to approach the performance problems, nor was I able to refactor the project to effectively have test and production environments reproducible. Claude did just fine.
It is 350k LoC, with over 138k as an Elixir app, 60k in tests/scripts, 65k Vue components, 45k JS, and more.
Coding is solved. Many people can't manage software engineering effectively. I don't know how much larger or how much more comparison is needed before this is accepted. Maybe it is "circular" reasoning like the dead comment says.
But it literally just works for me.
Every time. Every single time. People who say they never look at code anymore always say the trite things about "management of software development", how they have no issues, and that if you have issues, it's the problem with your skills with a side of snide remarks about your intelligence.
And then you press them for details.
After a few rounds of more and more trite answers they either go away or it turns out that their "automated everything" and "processes" and "coding is solved" are:
- a bunch of "make no mistakes" aka untold numbers of random markdown files aka skills, "guidelines", "memories", "ADRs", "technical docs" etc. Which are routinely ignored by any and all models.
- a bunch of tools that can't or don't catch all the mistakes the models routinely make. E.g. there are very few tools that reliably detect code duplication (especially code duplication with subtle changes in behavior), or reliably detect bad data access patterns, or reliably detect suboptimal algorithms etc.
- an overreliance on tests. The very same tests that are written by the very same very models
---
As a related aside.
I have multiple tools from linters to semantic analysis running as routine checks, and still. Here are the responses by Claude for some of the things I caught it doing in just the past few days:
- "The [test data] generator can sometimes fail, and create duplicated data, so I increased the number of times it runs to 100" (instead of fixing it)
- "The tests check this in an indirect way. They send megabytes of data and watch the memory of the whole VM. Unrelated processes change that number. The direct check is the number of body bytes that the server read. Do you want me to rewrite these tests" (Claude wrote all these tests to begin with)
- "The previous research covered only a few CSS files. It did not cover the design system documents, the components, or the pages that already use them." (so much for "No this isn't a magic prompt like 'make no mistakes'" as it had an explicit requirement to use the design system)
- "The too_large_message/1 helper has three copies, two of them private, with no good reason for them..."
And that's just some I documented for myself.
I don't use memories, markdown files, guidelines, etc.
I design the software like I would as an principal engineer and dictate the high level requirements. Opus 5.5 really does just work. 10 months ago this wasn't quite true, it required more steering of the technical requirements.
I don't know why your results are different. I run a live multi-week long campaign game that streams updates over WSS from an Elixir app. I can literally see if Claude is doing as it claimed. It is trivial to verify when it gets it wrong.
I dunno what everyone else is using to get such poor results.
It doesn't have a summary of the codebase. Or at least a useful summary.
> I design the software like I would as an principal engineer and dictate the high level requirements
Let me quote myself:
--- start quote ---
And then you press them for details.
After a few rounds of more and more trite answers...
--- end quote ---
This is now your third reply with trite answers (two to me and one to a sibling comment).
> I don't know why your results are different.
Because I actually look at code it produces.
> It is trivial to verify when it gets it wrong.
Yes. Yes it is.
E.g. in my codebase it is trivial to verify that it does 15 reads from the database on a page when just 2 or 3 would be enough. How do I know it? I check the logs, or see it in the REPL (I also use Elixir).
It's also trivial to see when it does shit like "oh I didn't read the design system". Because I actually see what it's doing.
It's trivial to see a bunch of other issues it routinely produces because tools like these exist: https://github.com/elixir-vibe/ex_dna and https://github.com/Cinderella-Man/credence (or even my own vibe-coded https://github.com/dmitriid/llamex and https://github.com/dmitriid/ex_css_analyzer)
> And if 350k LoC isn't enough for your use-case. Well, that wasn't quite achievable 12 months ago, but it was 8 months ago. In another year, I imagine if my codebase grew to 1M+ LoC, the latest models would still be keeping up.
Yeah yeah. "Keeping up". Where by "keeping up" you mean "I duplicated the code 15 times, and "I didn't read even half of what is needed to do this task".
It works for you because you give it only the most superficial tests. Wouldn't surprise me if you can cut all of that code in half and raise the fps 5-fold. But for that you actually have to look at the code.
----
BTW lol at the tests. Literally 10 minutes ago it produced this amazing test: https://x.com/dmitriid/status/2121723533567115357
I mean. Nothing wrong with it is there? The code works. I can see it work in my browser. What other checks do I need lol.
Have a fourth, then.
I hope you continue to ship for use-cases your users care about in your work, and continue to find satisfaction in software engineering.
I am.
All I can say is: there's a reason your software had multiple performance issues, couldn't support larger maps, and couldn't get past 20fps. It's the same reason why you're so impressed with the results and don't look at code anymore.
Adieu.
Every model, including $latest_model, makes mistakes and hallucinates things. If you aren't looking at the code, you are producing bad software, end of story.
> Its solved. Adapt and learn to use them effectively or get left behind; seems to be the unfortunate reality.
Ah, there's the FOMO push. It just isn't an AI bro post without false claims of "you're going to be left behind!".
I have automated the entirety of testing and Claude can validate it's own inputs. I then focus on marking requirements fulfilled, or bugs found. The bugs it does find are usually on the side of incomplete requirements. The few technical ones were due to a lack of testing. And I am not perfect, so I would have made all kinds of bugs myself.
If you keep to a mindset of, and build processes for, the management of software development rather than coding, results are better. No this isn't a magic prompt like "make no mistakes".
It is approaching the project with the mindset of knowing the limitations of the tools like one would a junior/mid engineer, and then mitigating. This is dependent on the codebase, product, and desired requirements for what processes to use and to what degree.
If you find yourself still needing to look, you have insuffiently automated the creation and validation of written software.
Its solved. Adapt and learn to use them effectively or get left behind; seems to be the unfortunate reality.