I implore you to look at it! I recently vibed too close to the sun: I knew at a high-level what I was doing, but never looked at the code.
Two weeks after and 50k LOC, I’m having a look and I can’t believe my eyes. There are whole failure modes that don’t need to exist. Moreover, how can I sale this software in good faith, when something happens and I’m largely a meat proxy because I don’t know how the code works?
I will go through the entirety of the code with my naked eyes, and make improvements (with AI).
But this time, I won’t fall for the illusion of “productivity”.
If you get into the regular habit of having the LLM look for issues with the code base it will find them: structural, bugs, design pattern issues, duplicate code, or same functionality not sharing a single path.
Honestly it feels like working with a real codebase. Tech debt always builds up, and yes the AI does it, too. It does notice if you ask to look.
When I wrote features by hand, I used to have like 2-5 bugs. Most of which were just human clumsiness.
When I started using AI and remained in the driving seat, most of the human clumsiness bugs went away.
Now, when I vibe, all I have are my verification loops and asking Opus5.5 to find and fix bugs.
Opus tells me it’s done. Only for me to start a new loop and it finds 10 more things or maybe I go in myself and it says “oh you are right”.
For me, this is not progress. I would rather spend an extra 1 or 2 hours a day in the driving seat building software that is correct and that is so simple that cost of changes is low.
It wrote them to begin with. Now you're handing of this discovery to literally the same LLMs that wrote this.
Oh, they will find some issues. Are you sure they will fix them and/or rewrite them without introducing more issues? How do you know if "looking is not necessary"?
I have automated the entirety of testing and Claude can validate it's own inputs. I then focus on marking requirements fulfilled, or bugs found. The bugs it does find are usually on the side of incomplete requirements. The few technical ones were due to a lack of testing. And I am not perfect, so I would have made all kinds of bugs myself.
If you keep to a mindset of, and build processes for, the management of software development rather than coding, results are better. No this isn't a magic prompt like "make no mistakes".
It is approaching the project with the mindset of knowing the limitations of the tools like one would a junior/mid engineer, and then mitigating. This is dependent on the codebase, product, and desired requirements for what processes to use and to what degree.
If you find yourself still needing to look, you have insuffiently automated the creation and validation of written software.
Its solved. Adapt and learn to use them effectively or get left behind; seems to be the unfortunate reality.
When I release features, players report minor bugs, which I can fix in minutes by simply dumping their complaint into claude.
I worked on this codebase prior to LLMs and made slow progress, with many bugs. I did not know how to approach the performance problems, nor was I able to refactor the project to effectively have test and production environments reproducible. Claude did just fine.
It is 350k LoC, with over 138k as an Elixir app, 60k in tests/scripts, 65k Vue components, 45k JS, and more.
Coding is solved. Many people can't manage software engineering effectively. I don't know how much larger or how much more comparison is needed before this is accepted. Maybe it is "circular" reasoning like the dead comment says.
But it literally just works for me.
Every time. Every single time. People who say they never look at code anymore always say the trite things about "management of software development", how they have no issues, and that if you have issues, it's the problem with your skills with a side of snide remarks about your intelligence.
And then you press them for details.
After a few rounds of more and more trite answers they either go away or it turns out that their "automated everything" and "processes" and "coding is solved" are:
- a bunch of "make no mistakes" aka untold numbers of random markdown files aka skills, "guidelines", "memories", "ADRs", "technical docs" etc. Which are routinely ignored by any and all models.
- a bunch of tools that can't or don't catch all the mistakes the models routinely make. E.g. there are very few tools that reliably detect code duplication (especially code duplication with subtle changes in behavior), or reliably detect bad data access patterns, or reliably detect suboptimal algorithms etc.
- an overreliance on tests. The very same tests that are written by the very same very models
---
As a related aside.
I have multiple tools from linters to semantic analysis running as routine checks, and still. Here are the responses by Claude for some of the things I caught it doing in just the past few days:
- "The [test data] generator can sometimes fail, and create duplicated data, so I increased the number of times it runs to 100" (instead of fixing it)
- "The tests check this in an indirect way. They send megabytes of data and watch the memory of the whole VM. Unrelated processes change that number. The direct check is the number of body bytes that the server read. Do you want me to rewrite these tests" (Claude wrote all these tests to begin with)
- "The previous research covered only a few CSS files. It did not cover the design system documents, the components, or the pages that already use them." (so much for "No this isn't a magic prompt like 'make no mistakes'" as it had an explicit requirement to use the design system)
- "The too_large_message/1 helper has three copies, two of them private, with no good reason for them..."
And that's just some I documented for myself.
I don't use memories, markdown files, guidelines, etc.
I design the software like I would as an principal engineer and dictate the high level requirements. Opus 5.5 really does just work. 10 months ago this wasn't quite true, it required more steering of the technical requirements.
I don't know why your results are different. I run a live multi-week long campaign game that streams updates over WSS from an Elixir app. I can literally see if Claude is doing as it claimed. It is trivial to verify when it gets it wrong.
I dunno what everyone else is using to get such poor results.
It doesn't have a summary of the codebase. Or at least a useful summary.
> I design the software like I would as an principal engineer and dictate the high level requirements
Let me quote myself:
--- start quote ---
And then you press them for details.
After a few rounds of more and more trite answers...
--- end quote ---
This is now your third reply with trite answers (two to me and one to a sibling comment).
> I don't know why your results are different.
Because I actually look at code it produces.
> It is trivial to verify when it gets it wrong.
Yes. Yes it is.
E.g. in my codebase it is trivial to verify that it does 15 reads from the database on a page when just 2 or 3 would be enough. How do I know it? I check the logs, or see it in the REPL (I also use Elixir).
It's also trivial to see when it does shit like "oh I didn't read the design system". Because I actually see what it's doing.
It's trivial to see a bunch of other issues it routinely produces because tools like these exist: https://github.com/elixir-vibe/ex_dna and https://github.com/Cinderella-Man/credence (or even my own vibe-coded https://github.com/dmitriid/llamex and https://github.com/dmitriid/ex_css_analyzer)
> And if 350k LoC isn't enough for your use-case. Well, that wasn't quite achievable 12 months ago, but it was 8 months ago. In another year, I imagine if my codebase grew to 1M+ LoC, the latest models would still be keeping up.
Yeah yeah. "Keeping up". Where by "keeping up" you mean "I duplicated the code 15 times, and "I didn't read even half of what is needed to do this task".
It works for you because you give it only the most superficial tests. Wouldn't surprise me if you can cut all of that code in half and raise the fps 5-fold. But for that you actually have to look at the code.
----
BTW lol at the tests. Literally 10 minutes ago it produced this amazing test: https://x.com/dmitriid/status/2121723533567115357
I mean. Nothing wrong with it is there? The code works. I can see it work in my browser. What other checks do I need lol.
Have a fourth, then.
I hope you continue to ship for use-cases your users care about in your work, and continue to find satisfaction in software engineering.
I am.
All I can say is: there's a reason your software had multiple performance issues, couldn't support larger maps, and couldn't get past 20fps. It's the same reason why you're so impressed with the results and don't look at code anymore.
Adieu.
Every model, including $latest_model, makes mistakes and hallucinates things. If you aren't looking at the code, you are producing bad software, end of story.
> Its solved. Adapt and learn to use them effectively or get left behind; seems to be the unfortunate reality.
Ah, there's the FOMO push. It just isn't an AI bro post without false claims of "you're going to be left behind!".
What I’m most anxious about is that, because I don’t read code, if I don’t have access to an agent for any reason I’ll have a hard time fixing a critical bug because I don’t have a mental model anymore of how things work. And that’s not professional and I can’t sell that software.
I think this is a growing problem. I pulled back. I'm much faster than I used to be. BUT, I also forcibly insert myself into the loop. I've thoroughly iterated on a design with the LLM before starting. I've personally looked at and often iterated on every line of code before merging. I regularly walk through the generated code base, both with and without LLM assistance, in order to ensure I have a solid mental picture of what's there.
This behavior has been informed by seeing things go to hell, both for myself and with other people, clients, open source projects, etc
When you have a deep understanding of something, you just can’t help thinking about all the ways you could improve it. Maybe you realize it could be generalized, or that a different problem you’ve had could be solved with a similar technique. Sometimes you get oh “OH DUH” moment where you realize you were making the slightly wrong thing this whole time, and NOW you know EXACTLY what to make.
With LLMs I just get basically what I asked for, and then I’m done. Obviously this is sick and I love it.
But for a lot of my projects recently, I fall into this trap
- I have a problem, I think a program could help. But I want to know if there’s a viable approach before I invest a bunch of time by myself, so I boot up claude
- LLM writes the entire program, I don’t know what’s going on in there
- the program kinda solves the problem or it doesn’t. At this point I already have something that has vaguely confirmed or denied the viability of solving the problem, so I think “welp I’m not gonna spend a whole week of my life reading an LLMs output”, and that’s that
I rarely have cool insights about the problem after that. In the olden days, even if my approach didn’t work, I would come away with a new understanding of that problem, and all similar types of problems. “Oh that’s why people say cache invalidation is unsolved”. “Oh that’s why generics can’t infer xyz”
Don’t really get that when you vibecode. But the siren song of just having something that works is so sweet, and I can never ever resist it
This! Especially this! is the real reason I started building my own commercial software again.
I’m betting that the details will still continue to matter.
Absolutely true! Some month ago I discovered huge memleaks by coincidence because Opus nailed tons of LINQ calls together and duplicated data as hell.
It is. And the trouble is, once you do that, the speed gains evaporate. It takes just as long to carefully review the LLM's code as it would've to write it yourself, because the typing wasn't what took time, but the understanding of the problem. The only way to get productivity gains from an LLM is to vibe code, at which point the quality of your software craters.
"Technical debt is a qualitative description of the cost to maintain a system that is attributable to choosing an expedient solution for its development.
While an expedited solution can accelerate development in the short term, the resulting low quality may increase future costs if left unresolved."
AI is right now buying down huge amounts of technical debt (see the flurry of CVEs). But we're already onto the self-improvement treadmill: future more capable systems will be better at solving any issues introduced today - it's literally not worth the time worrying about now (this is the same reason you don't get a cancer prognosis over a longer then 10 year time frame - the expectation is future treatments will be better).
The technical debt my code written in Opus 5.5 might have is already going to be handled by Fable 5.2, for example. It's going to be better and faster then I am. But also: if the rate of technical debt production from AI is even slightly lower, then it gets more and more manageable in a future where nothing changes but the increasing volume of AI compute is concentrated on less and less of it.
I mean certainly it would be nice if I could implement a feature without Claude finding a security bug and tripping cyber protections every couple of hours.
No, we can't. We can believe this will be the case
> The technical debt my code written in Opus 5.5 might have is already going to be handled by Fable 5.2, for example.
"Let's generate more and mire garbage in hopes that there will be some magical tech that solves the issues" is not that good of a strategy.
E.g. even for the amazing magical benchmark-shattering Fable 5.5 Anthropic's guidance is still the same: your CLAUDE.md should bot exceed 200 lines and claude only reads 200 lines of memory index because otherwise it quickly loses adherence.
Like all problems, we'll solve them when we get there but its not like anyone has an abundance of useful alternative advice: I'm getting useful software out now, whether I have to fix it later has the same problem it has always had - when we need to, we'll do it.
It's already here
> its not like anyone has an abundance of useful alternative advice
Yes, the advice is the same: look at the code produced and understand it.
> whether I have to fix it later has the same problem it has always had - when we need to, we'll do it.
Yet you don't. You're quite literally saying "who cares some future model will solve this for me".
But the model I buy today is cheaper in terms of capability then the one I could've bought a year ago.
But software can be "done". There isn't an endlessly acruing technical debt, the product can be finished.
Yes my frontend is nicer. I do not hang myself up debugging a small issue for hours. I also don't have to search questions online, and get even complicated answers fairly quick.
But I'm not sure if I ended up being so much productive, or I'm just building a lot of open source stuff nobody needs. Last year i would have solved a narrower problem and relied on others to contribute the rest.
Why hope for a resurgence of indie software without also a revolution in enterprise?
If there ever was a time for either entrenched players to improve their UI/UX (using LLMs) or for rival upstarts to build lots of feature parity, now seems like it would be the time.
Perhaps if Agents are given some autonomy to buy enterprise software trials it would be easier to break the stranglehold of the biggest vendors.
Where I'm finding AI exceptionally useful is now being able to address tech debt and developer experience for my team WITHOUT having to argue for spots in the backlog to do it.
So, if nothing else, it sucks a lot less to work on enterprise software if you can let a team lead or principal engineer loose from the PM to just fix things.
But, how is AI fixing this? Unless the perverse answer is: no one’s reviewing code anymore so I can just merge my technical fix.
So "Where does this function get called from", "What does this function do?", "Rewrite this function as-is with better variable names", "Make a document outlining the structure of this project", or "Review this PR and watch out for any places that may be lacking the fix" are all reasonable things to ask of an AI.
When you're in a big project and aren't sure where this is supposed to go or how things get A from B, AI can be extremely helpful.
In which sector is your company active where this is allowed? Or do you run on your own infra?
Yes, I imagine this is trickier in more sensitive applications, but I'm sure this isn't a new concern. AWS has Private Cloud, surely you can get a LLM with whatever assurances a more sensitive project needs at this point. But in my current environment that's not really a concern, so I've not personally looked into it.
Nit: not all of those questions require statistical language models. IDEs like IntelliJ have been adept at syntactical queries and refactors for many years using purely deterministic data structures and algorithms.
Like if you have some code that's been intentionally obfuscated, or made by somebody who names everything "a", "b" and "c", a LLM is likely to be of at least some help.
And yeah, you can do things like finding all the places where func() is called, but a LLM can also be helpful in saying where something needs to be done, which is a much fuzzier task.
Usually, If you don’t know that, you shouldn’t modify the system either. Just because you know how to unscrew a bold, you shouldn’t go about removing the head cover of an engine. Locating the area of interest takes a lot less time than actually understanding how it integrate, the guarantees provided by other systems and the guarantees that it offers.
Modularity in software architecture. By reducing the links between parts of the code, it’s easier for someone else to navigate the system and answer these questions. But it does require careful design and caring about the code.
In my own projects I'm also quite capable of forgetting stuff. Double checking with a LLM whether I forgot about one of the more obscure corners doesn't hurt.
That can be a sign that what you're now building is perhaps not worth building? At least my experience from dayjob, we're building more and faster, but it's either things we really shouldn't do (stuck in backlog forever for a reason), or things that just complicate the main product when features get added willy nilly.
Of course this calculus drastically changes when maintainability and other stakeholders come into the picture. There vision an architecture are more important than adding features. Architecture isn't too hard to solve, but sadly good product owners have always been hard to find
I have the opposite view. Why would anyone build and sell software now when anyone can one shot anything? Genuinely interested in people’s thoughts.
The value added by commercial software will have to be much higher than what it is now, or most companies will have little reason not to code their own, fully customized to their own needs.
But it is really hard to imagine what the future of software will be, even in 2/3 years time. We live in wild times.
A company usually wants support / integratoin / maintenance / updates / new features etc. - for sure there will be some companies building 100% on their own, and yes I guess this number will increase but I do not see any mid-level company or above replacing Oracle or Salesforce or Microsoft with their homegrown solution.
Which is to say, the form of your statement I'd agree with is: if anyone can make an MVP in a few days, what use is an MVP?
A church is going to pilot it to 450 people. Some people actually do find it useful.
https://jaisenmathai.com/articles/sojourn-for-ios-was-45-one...
And the fact that some apps can be made in one prompt doesn't mean every app can. I've been working on a fully AI generated project for 8 months, it's taken billions of tokens and continuous prompting. The amount of human time spent on this project is just as much as I'd spend before AI, just now the project is much larger in scope. So even if some people have the ability, they still might not want to spend their precious human time on it.
I agree that the ease at which it can be created feels a little…how do you say it…demoralizing.
In the long run, I don’t think this will be a net positive. But in the scope of this conversation about absolute productivity and code quality I seem to disagree with a lot of HN.
I think marginally is an understatement though because it feels like it is compounding. I don’t know other people’s experience but I’m building software that would not have existed before LLMs and coding agents arrived.
And this software is without a doubt better from the end users perspective. Maybe the code is crap, who knows? I don’t look at it. I judge based on 25 years of writing software by hand, deploying it to real users, and being a user myself.
I hope there is a resurgence of indie software like existed in the early 2000s.
There’s no hope for enterprise software. It will always be terrible.