Re:AI usage generates a fragile codebase
by jansport123 on Tuesday November 10, 2026 @06:25AM
(#50438795)
we'll recalibrate eventually based on understanding what level of abstraction we can operate on. If these code bases become unmanageable then the trend will correct itself.
Re:AI usage generates a fragile codebase
by n4r9 on Tuesday November 10, 2026 @06:27AM
(#50438815)
I can't imagine a broad movement of devs using AI more thoughtfully without a widespread incentivisation from employers. And I can't imagine the latter at all. We'll probably just have to cope with more brittle/fragile codebases.
Re:AI usage generates a fragile codebase
by n4r9 on Tuesday November 10, 2026 @06:51AM
(#50439004)
It's tricky when it comes to performance evaluation. Metrics like token usage or story point "velocity" are easy to conjure up even though they mean little. The kind of careful work that goes into building robust, maintainable code drags such metrics down, and it's very hard to quantifiably gauge their positive effects.
Re:AI usage generates a fragile codebase
by duskdozer on Tuesday November 10, 2026 @07:07AM
(#50439161)
I doubt this if the patterns of tech/software over the last <x> years remain. We'll just get buggier, worse-functioning, more expensive products/software built by fewer, more burned out developers. But your manager will love the sales presentation, so all will be well.
Re:AI usage generates a fragile codebase
by tarkin2 on Tuesday November 10, 2026 @07:01AM
(#50439094)
It seems obvious--and correct me if I'm wrong--that using AI requires less mental effort; and a less exerted mind is a weaker mind. A weaker mind will continue to create more and more fragile code.
Re:AI usage generates a fragile codebase
by thisisnotgreat on Tuesday November 10, 2026 @07:24AM
(#50439333)
Well, it's a nice thought-stopper by now, but IME it requires more and I can see the oldschool devs struggling with it every day.
Every once in a while I go "manual". It's restful and entertaining to write syntax by hand. It has this peculiar dare I say autistic delight, sorting the colored blocks and getting a little kick out of it. I understand the nostalgia but the progress is breathtakingly slow - as expected. You cannot do 6502 assembly by hand and expect to be fast in any meaningful sense of the word.
I usually juggle four, five active projects at a time and make days worth of manual progress on them in a morning. Support calls and colleagues questions interject occasionally. It's much more work than before, but I like it because dev work is super boring and now it's getting more interesting.
Like many people try to say AI is raising the bar. What we call normal software now will be childrens' toys tomorrow. The world expects ambitious projects, stuff that is hard and stands out.
Re:AI usage generates a fragile codebase
by CharlieDigital on Tuesday November 10, 2026 @06:27AM
(#50438809)
it doesn't have to.
AI is great at following patterns and loves to write tests. Start with good patterns that most teams would not have the discipline to write by hand and it is possible to get very robust code.
Re:AI usage generates a fragile codebase
by CharlieDigital on Tuesday November 10, 2026 @08:32AM
(#50440059)
AI will write tests to the level of depth that you ask it to.
Try:
- Clearly identify the invariant being tested focus on exercising the invariant
- High volume, low signal tests increase CI and build times
- Avoid low signal tests; raw coverage is not the objective
- Avoid write-then-read tests when risk is low (no transformations, no business logic in application code on write or read path)
What's useful is to have a second agent that reviews the tests impartially and trim the ones that are low signal.
For our tests, we also added a small helper library[0] that cleans up tests to make them easier to read (smaller context for both humans and agents) as well as to write. Agents write average code by default. Your job as an engineer in this age is to help agents write good code.
Re:AI usage generates a fragile codebase
by soon_to_be_sb on Tuesday November 10, 2026 @06:29AM
(#50438829)
Every once in a while something comes up that proves beyond a doubt that 99% of human beings on planet earth cannot think for themselves. And most sensible things that human beings do, they do by following the herd or "instructions".
Re:AI usage generates a fragile codebase
by soon_to_be_sb on Tuesday November 10, 2026 @08:40AM
(#50440193)
Am I?
Tell me again where there is NO widespread automatic consensus about how imperative is that one HAVE TO use these things from a sandbox?
The last time this happened was when the whole world agreed that it is alright to take a device with multiple cameras, mics AND 24x7 internet connectivity to our most private places...
Re:AI usage generates a fragile codebase
by ricardobeat on Tuesday November 10, 2026 @06:33AM
(#50438848)
Imagine applying this to a particular junior engineer:
- they write shallow tests
- they write overly complex code
What do you do? You provide guidance, support and tooling so they can write better tests and code. It is the same with AI agents (plus the hassle of how to carry that knowledge around and transfer it). You are responsible.
You are also responsible for the understanding part: know what is being worked on, read/skim the code, inspect the outcomes, cross-check specifications, tests, product requirements. The 'moving too quickly' idea is the other side of the same coin, it is your responsibility to keep track of where the project is going. The same thing happens in a project where you have 10 developers (or 200), you cannot possibly read all of the code they write.
Re:AI usage generates a fragile codebase
by ako on Tuesday November 10, 2026 @06:49AM
(#50438986)
If you are a bad senior, yes. If you are a good senior you train the junior so his output will improve and you train the agent so its output will improve.
Re:AI usage generates a fragile codebase
by cassianoleal on Tuesday November 10, 2026 @06:47AM
(#50438972)
When I guide a junior engineer, I'm helping a human being become a better version of him/herself, who also helps me grow as a human being and will contribute with their unique ideas and world view, to the team, the organisation, the industry and society.
When I do the same with an LLM, I'm just repeating the same guidance over and over again out of frustration just to get something done.
Comparing an LLM to a junior engineer is frankly a disgrace.
Re:AI usage generates a fragile codebase
by ricardobeat on Tuesday November 10, 2026 @08:06AM
(#50439761)
How do you suppose project leads, principal engineers, EMs provide guidance? You don’t have to read the code being written by entire teams/departments for that.
Re:AI usage generates a fragile codebase
by ernst_klim on Tuesday November 10, 2026 @08:43AM
(#50440234)
They don't. They manage highly autonomous very well educated senior engineers, who can do the work autonomously. LLMs can't, they are closer to interns with dementia (and occasional genious revelations).
Re:AI usage generates a fragile codebase
by hhjinks on Tuesday November 10, 2026 @07:10AM
(#50439183)
That was how I understood "code was never the bottleneck". If I need a deep understanding of the code, I might as well write it myself. It won't increase the time I need for the task appreciably.
Re:AI usage generates a fragile codebase
by gonzalohm on Tuesday November 10, 2026 @07:45AM
(#50439570)
Exactly, same thing with considering AI a higher level language. If I have to think of all the corner cases and design choices in plain English, then I much rather use a programming language whose main purpose is exactly to define cases and behavior of your design
Re:AI usage generates a fragile codebase
by menaerus on Tuesday November 10, 2026 @07:22AM
(#50439319)
Yes, it does, much more than the person sitting next to you or yourself. I wrote probably 1-2 MLoC in C and C++, and I literally see no reason why would I not be using the AI to produce code when it's already much better than I am, and much better than pretty much anyone I know or have worked with.
Re:AI usage generates a fragile codebase
by dofm on Tuesday November 10, 2026 @06:58AM
(#50439080)
An LLM can't have intuition itself.
It can only simulate the outcomes of intuition in its training set.
The likely uncloseable gap is that some intution is unspoken, literally felt by the brain's default mode network.
LLMs basically never shower, do the washing up, kick their foot in the dirt while sitting on a park bench with a coffee: the domain of the default mode network.
Re:AI usage generates a fragile codebase
by amelius on Tuesday November 10, 2026 @07:12AM
(#50439202)
> LLMs basically never shower, do the washing up, kick their foot in the dirt while sitting on a park bench with a coffee: the domain of the default mode network. And they never sit frustrated looking at things, guided by their long experience, with the itchy brain feeling that there is a solution before they can even say the words.
I'm sure the training data of LLMs contains a lot of those things, though.
Re:AI usage generates a fragile codebase
by dofm on Tuesday November 10, 2026 @07:19AM
(#50439273)
It contains the averaged written ghosts of the outcomes of those things.
But they are not the things themselves — they are incomplete representations; just words that approximately describe the application of that intution to the problem at hand.
Think about how you visualise complex code projects in your mind — how it's not quite diagrams or buildings or words but multiple things in a sort of multi-dimensional, itchy space that is satisfied when a really good implementation appears. Think about how often the code you have built retains that wordless shape long after you wrote it down. Think about how impossible it is to adequately describe it.
Re:AI usage generates a fragile codebase
by Leynos on Tuesday November 10, 2026 @08:02AM
(#50439732)
Coming soon: language models trained on 2000 years of tactile, proprioceptive, spatio-temporal, calorific, olfactory, gustational, humidity, temperature and barometric data.
The downside: the model doesn't always have words to describe what it's feeling.
Re:AI usage generates a fragile codebase
by krige on Tuesday November 10, 2026 @07:38AM
(#50439479)
It may contain writings about these things, not the actual things. That's more like saving a jpg at high compression over and over if you want catchy metaphors.
Re:AI usage generates a fragile codebase
by dofm on Tuesday November 10, 2026 @08:01AM
(#50439719)
FWIW, I guess it probably doesn't always matter too much in a tool where you want a logical, correct, standard, robust outcome like agentic coding.
Those outcomes are more easily averaged and we are after all talking about artificial intelligence.
I just think it's important to note, every now and then, that words aren't feelings or thoughts, and that an author's next book cannot be calculated from only the words of their previous books, etc.
Re:AI usage generates a fragile codebase
by kelseyfrog on Tuesday November 10, 2026 @08:08AM
(#50439792)
Humans still havent't accepted The Bitter Lesson[1].
Compute > Insight. Every time.
We can keep searching for essential human characteristics that humans have and AI lacks, but every attempt increasingly resembles an ego-preserving negotiation with the inevitable march. It's reminiscent of the God of the Gaps[2], but for human exceptionalism. As AI competency encroaches on more and more of human ability, we'll retreat into smaller and smaller gaps - foxholes of inimitablity.
As long as human value is justified by ability, quality, or characteristic, this problem will endure. The one escape hatch is the classical liberal idea that human value isn't tied to anything, that, by virtue of being human, people have universal, inalienable worth.
We should stop the project to attempting to substantiate human value by other means.
Re:AI usage generates a fragile codebase
by cyanydeez on Tuesday November 10, 2026 @06:33AM
(#50438857)
The silent part is "undirected". You can patch these holes by observing them and the author us aware. By itself, these are the flaws.
I think speed is the biggest flaw though. If i could get variable speed while planning vs building, thatd change some things. I run local models, so i am mostly in the slow lane. When patching these flaws, i want it to read the code parts and diagnose quickly.
When its coding, i want to see the problems its encountering with the prompt. I usually observe its thinking a few rounds and if it misunderstands, i redirect.
But in all of this, i want a dial that just speeds up once i feel its doing the right thing.
SOTA today, to me, would put a model floor like qwen3.8-flash-next on a platform i can dial up the speed. Thats all. I think many of us will bring our own harness, so the only hard point is how dumb the engineers tools are.
Since im building local, my tools do a lot more management. So.e of tge opinions i see on HN trigger a "skill issue" reflex because we have a whole shopping mall now if tools and the incurious seem oblivious to how these models are used by cloud providers.
Learning about the raw capabilities of LLMs makes getting them to "do the right thing" is now just a job description.
Lastly, the last decade of tech oligarchy exposes how little education elites had about humanity and wicked problems. Elon musk reinventing public transportation several times ignorantly. The fact that a social program requires the consent of the people it helps. LLMs are a stark lesson in the difficulty of knowing all the constituent parts and still being unable to predict outcomes.
Re:AI usage generates a fragile codebase
by fragmede on Tuesday November 10, 2026 @06:47AM
(#50438968)
> Elon musk reinventing public transportation several times ignorantly.
In interviews, he admitted that the Hyperloop was intended to mess up the California High Speed Rail project because trains are boring. It's the opposite of ignorance, it's malice. Hanlon's inversion.
Re:AI usage generates a fragile codebase
by Topfi on Tuesday November 10, 2026 @06:36AM
(#50438879)
I do agree with some parts but not others. Reverting a flawed changed, e.g. should have never been as difficult as I have seen it been in some orgs before the advent of LLMs. Same for poor UX landing, both are issues that sure, may be thrown into sharper relief now, but were issues in some code bases and not others and remain that way today. More over, they are issues which can be permanently addressed in the former and in the later are one of the key sources of differentiation today.
On code readability, shallow tests and general quality, I can agree emphatically, though not universally. In other words, there are models which write the densest, hardest (for me) to parse code and tests that break with the lowest load/timing deviations yet proof less than nothing.
These do that regardless of how much one ships in my experience however so I wouldn’t attribute this to the velocity and more poor data at the heart.
I know this reads a bit like “you are using the wrong model” and I understand if someone testing e.g. an OpenAI models understandably extrapolates across all models and considers poor parse-ability in code a likely result of velocity, but you can always step back and work slower, plus, for what it’s worth, I struggle to parse OpenAI code when running their models through my evals after step 7 which is still a fairly measured in scope code base.
Essentially, I have seen LLMs, even the newest frontier, produce barely legible slop when building simple CRUD and I have seen a massive rewrite of the existing Firefox code base using APZ to enable an infinite canvas of browser tiles grow in a manner where, even at this stage having replaced a lot outside Places and Gecko, I can walk through changes made and explain the logic, as well as look at any part I haven’t been in for a while and find my bearing even without assistance.
Re:AI usage generates a fragile codebase
by snarfy on Tuesday November 10, 2026 @06:39AM
(#50438897)
Business incentives create fragile codebases.
AI is a force multiplier in whatever you apply it to. Most software companies output binaries with new features, whether it runs locally or on a server. Binaries with new features. None of the business owners give a shit about what is in the binary. It may be extremely fragile and only work for one type of user, but if they only have one type of user nobody cares. Most software doesn't keep evolving forever and at some point gets rewritten as a 2.0 version or whatever, so the quality was never as important.
It doesn't have to be that way. You can use AI just as effectively to produce quality source code as you can slop binaries. It depends on the business priorities and what is actually important.
The caveat is if you are the one engineer on the team trying to use AI the right way to produce quality source instead of binaries as fast as possible, you might get left behind or let go since your work no longer aligns with the business.
Re:AI usage generates a fragile codebase
by user43928 on Tuesday November 10, 2026 @06:41AM
(#50438912)
After 7 months and a 500k LOC codebase I can say that in my experience the fragility is inconsequential.
Disclaimer: I have no SaaS platform, it's a mobile app.
In my experience maintenance does not suffer long term with codebase size, if anything it became easier with model advances.
Most people talking about suboptimal implementation, fragility, and maintenance burden seem to theorize and extrapolate with no actual experience in implementing a large project with AI.
I can't blame them, I see the same in my day job, but the results after 500h+ on a separate project show that the concerns do not manifest in practice.
Re:AI usage generates a fragile codebase
by glimshe on Tuesday November 10, 2026 @06:52AM
(#50439008)
Tell us more. What I'm gathering from your comment is that you have a personal monster project (500k LOC) fully/mostly developed by AI and you're having no problems maintaining it. Can you understand anything in the codebase? Has the maintenance cost (in AI tokens) been going up as the project keeps growing? Are users happy?
Also, sharing a link would be great if you're comfortable with the idea.
Re:AI usage generates a fragile codebase
by user43928 on Tuesday November 10, 2026 @07:06AM
(#50439159)
No, I'm not comfortable linking my HN account to my name and home address.
I'm still in the process of launching the app, it went live just two weeks ago. So far only 30 users, I have not yet started the ad campaign.
Basically I don't look at the codebase at all, I have long stopped using the IDE or doing code reviews.
From a product perspective, there have been no big surprises. Only once I was surprised to see a feature I thought I implemented only on a specific screen being available on a different screen too, hidden behind a drag gesture, spawning an UI component I totally did not expect.
While that seems like a loss of control, it's not too different from enterprise work with many contributors where I am also not aware of all changes.
About maintenance cost, no, I do not think so. It generally became easier over time since Opus 5.5 delivers better results than GPT-5.5 did.
From a token perspective, a large part of the resources go into translating to 30 languages and checking the screenshots for layout regressions.
Re:AI usage generates a fragile codebase
by hologramofham on Tuesday November 10, 2026 @07:52AM
(#50439633)
> You think that after my 500h of work and daily usage of my own app, I have no idea about what maintenance looks like.
At 30 users you have no idea what maintenance looks like. That's literally what maintenance is - responding to real world usage.
Come back with your experience when you have lots of users that stick with your app, you're putting out fires and implementing new features. Your experience will be meaningful after 6 months to a year.
I'll wager that right now you have a PoC that needs a lot of work to be something that retains users. That's not a criticism, that's just how it works for every startup.
Re:AI usage generates a fragile codebase
by user43928 on Tuesday November 10, 2026 @08:04AM
(#50439750)
I have been QA testing, fixing bugs, and adding new features near full-time for the last seven months.
I have a pretty good idea about what maintenance looks like. Particularly so about the technical aspects of how effective the models are at refactoring or fixing bugs.
Even if I were to accept the argument that users would have entirely different expectations for my app than I do, it hardly changes the fact that I already know I can easily implement large changes in the code base.
Re:AI usage generates a fragile codebase
by thisisit on Tuesday November 10, 2026 @07:45AM
(#50439569)
Tell me you know nothing about maintenance without saying you know nothing about maintenance. The best part is that you call others about not knowing what they are talking about.
Come back after running your mobile app for at least 6 months in the real world with real users and then we will talk about maintenance.
Re:AI usage generates a fragile codebase
by jvidalv on Tuesday November 10, 2026 @06:53AM
(#50439026)
Same, I maintain a really healthy mountaineering app (6K users), I started it where we where still pasting into chat-gpt, and now fully autonomous with Claude Code.
Codebase has never been better and healthier, and any tech debt that has been accumulated after a few releases can be removed in a few prompts when you are out of product ideas and have some free tokens to spare.
Re:AI usage generates a fragile codebase
by Topfi on Tuesday November 10, 2026 @06:55AM
(#50439042)
Purely for the models running your code base, have you ever ran an assessment of what that fragility costs you? Tool call error frequency, unnecessary test failures that the model then addresses but wouldn’t in a more parsable code base, inefficiency in navigation, I have seen all that when running a fairly new eval inspired by something I read in the ts-rust repo, giving a simple GPT-6 Astra code base to Opus 5.5. Implementation accuracy was higher and token usage lower when Opus 5.5 got a Fable 5.1 built code base of the same feature set and coverage compared to the fragile GPT one. Incidentally I struggled to parse the latter but not the former.
Yes, a modern model will happily work on the most unmaintainable slop (doesn’t even need to be LLM derived slop, we have all seen bad Human written code) and that is great if you have an old code base to maintain. But if one can avoid it, even if one uses only models and never looks at the code, being efficient has value. Heck, rewrites are doable and affordable now, I have restarted projects and the gains were quickly apparent, even if we just look at model usage efficiency gains.
Re:AI usage generates a fragile codebase
by user43928 on Tuesday November 10, 2026 @07:26AM
(#50439353)
I started the project during 'copy snippets from ChatGPT web' days, so technically not entirely generated, but I don't think much is left of those days.
Re:AI usage generates a fragile codebase
by nahumfarchi on Tuesday November 10, 2026 @06:57AM
(#50439061)
I recently spent a couple of hours reducing 1k lines of code produced by an LLM to about 100. Purely extrapolation, but imagine what if your project was 50k instead of 500k?
Re:AI usage generates a fragile codebase
by madaxe_again on Tuesday November 10, 2026 @06:58AM
(#50439078)
I have a similar position - maintainability of code written by AI no longer matters in any meaningful way, as as the models advance, one simply rewrites.
At the current pace, the very idea of maintaining software will become as relevant as the idea of trimming wicks, stropping razors - or making every frame you take with a camera count.
When the marginal cost of replacement drops to a sufficiently low level, the entire paradigm changes.
Re:AI usage generates a fragile codebase
by rsfern on Tuesday November 10, 2026 @07:33AM
(#50439432)
Interesting analogy choices. Aren’t straight razors still the highest quality shave? We go with electric and cartridge because it’s faster and cheaper and takes less skill to get a decent result
Digital cameras are another interesting one. Since capturing images is so cheap we just snap a bunch of frames without a lot of thought towards composing them carefully, with a side effect that we get an overwhelming amount of mediocre frames and a few good ones, but we have to sift through.
I’m still not convinced I want to make this tradeoff in programming. But I work in scientific computing where I think it’s (still) really important to understand what modeling assumptions you are making through code
Re:AI usage generates a fragile codebase
by madaxe_again on Tuesday November 10, 2026 @07:43AM
(#50439545)
Yup - I think there will remain a space for artisanal code for a decent while yet, particularly in things like scientific computing, safety critical systems, and similar - but for most purposes, the cartridge shave will be good enough, more convenient, and much, much cheaper.
Re:AI usage generates a fragile codebase
by quietbritishjim on Tuesday November 10, 2026 @07:16AM
(#50439250)
> In my experience maintenance does not suffer long term with codebase size, if anything it became easier with model advances.
This reminds me of Moore's law in the 1990s.
If you program run slowly, or used too much RAM, you could spend six months optimising it (or removing features). Or you could just sit on it for six months and release it as it is - hardware moved on so fast that, by then, it probably ran successfully.
Your suggestion is a bit like this: don't worry, the pace of model understanding will catch up faster than the code will get disorganised.
I'm not saying I agree but it's an interesting idea.
Re:AI usage generates a fragile codebase
by thatsabadlook on Tuesday November 10, 2026 @07:40AM
(#50439502)
Nah, Ive seen sloppy code bases. They really are hazards to the operations and companies they support. I'm not anti ai, but people opting to lose complete control of their own products so there engineers don't have to think are suffering from it, and it will get worse and worse and worse.
Re:AI usage generates a fragile codebase
by beastman82 on Tuesday November 10, 2026 @06:44AM
(#50438952)
Try using a statically typed language, refined types, and functional programming. Your tests become limited to things that are hard or impossible to assert with the type system.
Re:AI usage generates a fragile codebase
by sham1 on Tuesday November 10, 2026 @07:13AM
(#50439216)
You can't hold an LLM accountable, that's a big one. Nevertheless I foresee some people trying to shift blame to the LLM if something breaks. "Claude told me..." and so on.
Re:AI usage generates a fragile codebase
by onion2k on Tuesday November 10, 2026 @07:22AM
(#50439318)
I haven't ever seen a human developer inventing requirements.
My first startup was building requirements management software for small businesses to solve that exact problem. Devs add things in all the time. It's usually referred to as 'gold plating' or scope creep.
Re:AI usage generates a fragile codebase
by onion2k on Tuesday November 10, 2026 @07:20AM
(#50439285)
There's an implicit assumption that humans will get things wrong, which is why we invented pull requests, code reviews, design sessions, unit tests (with separate reviews for those), etc.
The belief that AI can do all that without the gates is throwing away something useful where people get to understand how the system is changing. If you keep the gates but use AI to generate all the code you either slow AI down to the speed of humans or you end up with a massive backlog of stuff to review.
Re:AI usage generates a fragile codebase
by hologramofham on Tuesday November 10, 2026 @07:40AM
(#50439510)
> How's that different from a regular human dev doing the same thing?
Another of the shallow arguments thrown around, similar to "we don't check compiler output so..."
People learn and improve. LLMs don't learn, they follow instructions to a degree.
A human has to wrestle with it to work how you want, ergo humans are necessary already.
You can't fire the LLM and hire a competent one. The few available is all there is, they're all copies of the same thing.
You don't have a team. If you think 10 LLMs = a team of 10 people, then you don't know what a team is, how different skills and views make a big difference to outcomes.
LLMs can't innovate. They can't come to you with a great idea they came up with. They don't have ideas.
If none of these things are worth anything in your particular case, then that's ok. But let's not actually lose our minds and forget why people are intrinsically better than machines.
There are only a few arguments to support using LLMs, all of which are from people who a) want to spend less on salaries or b) sell you LLMs.
Re:AI usage generates a fragile codebase
by hoppp on Tuesday November 10, 2026 @07:55AM
(#50439664)
LLMS definitely improve but it's the model updates.
Teams are often hierarchical where the leaders define everything and people under them have less things to say about it. Not much different than an LLM team.
Re:AI usage generates a fragile codebase
by sausagefeet on Tuesday November 10, 2026 @07:32AM
(#50439416)
We do this with OCaml, and it works pretty great. I have a particular style that I like and it takes some working of skill files to ensure it adheres to it, but in terms of just writing OCaml, despite being a relatively unpopular language, I have not had issues in any model that I've used.
Re:AI usage generates a fragile codebase
by skydhash on Tuesday November 10, 2026 @07:03AM
(#50439124)
> Your tests become limited to things that are hard or impossible to assert with the type system
Type systems are not a panacea. You still have to be logical and careful about them, especially around boundaries where types are just assumptions and not logical inferences. Then you have functions that switch type, but only a subset of the input is actually valid (string -> int for a number parser). Checking for NaN is like checking for Null.
Type systems does help, but they need the same attention as code to be actually helpful.
Re:AI usage generates a fragile codebase
by stingraycharles on Tuesday November 10, 2026 @07:06AM
(#50439157)
I feel this is a nice theory but needs even more ruthless oversight in order for the LLM to pick the correct abstractions and types, as your tests are now effectively your higher level type system choices.
Re:AI usage generates a fragile codebase
by drooby on Tuesday November 10, 2026 @07:21AM
(#50439296)
Step 2: Prompt the agent to comment out the "when" code and check if the tests fail. (If tests don't fail it's a poor test)
Step 3: Prompt the agent to write mutation tests to harden the suite
Step 4: Prompt the agent to write property-based tests.
Step 5: Prompt the agent for 100% code coverage
Step 6: Keep a look out for small issues with steps 2-5 above and refine a Skill or Rule to package the workflow and plug gaps in the model's current shortcomings.
Step 7: Dedicate some time to manually reviewing and thinking about interesting edge cases worth testing.
---
If you follow these steps, you will have better test coverage than 99.9% of any code an engineer ever wrote pre-AI... and it will be almost completely automated.
My team's incidents and bugs are DOWN this year while we're shipping more.
Leveraging AI effectively requires a new paradigm of interaction. The name of the game is to use techniques and strategies and guardrails that create an environment that produce good software.
Re:AI usage generates a fragile codebase
by stingraycharles on Tuesday November 10, 2026 @07:33AM
(#50439428)
It’s also a great way for AI to keep creating backwards compatible bloat code when refactoring and whatnot because “tests are failing” when those tests are testing absolutely silly stuff (LLMs have a tendency to want to validate exact output rather than behavior).
I don’t think anyone has figured out the proper solution yet, I personally try to get the LLM to write code similar as I would have written it myself.
Re:AI usage generates a fragile codebase
by drooby on Tuesday November 10, 2026 @07:47AM
(#50439597)
What do you mean "exact output rather than behavior"?
If you mean asserting against hardcoded literals, then that is the preferred testing shape in most cases. And for good reason.
If you change production code and a unit test fails, that is useful. Idk why that's silly. If you want to test more complex interaction write a UITest or integration test.
I honestly think we're all living in different worlds. There's probably large differences in capability on a per-language, per-team, per-codebase, per-engineer environment that are producing wildly different perceptions in output quality.
Re:AI usage generates a fragile codebase
by stingraycharles on Tuesday November 10, 2026 @08:08AM
(#50439784)
It would add tests for exact error messages, for example. It would add tests that the process exited with very specific error codes. Basically tests that I would never ever write myself.
And yes everyone works in different domains. I work at a database vendor myself, so mostly lower-level work.
I’m very happy and successful with LLMs, I just don’t think that these “just move all your tests to your type system!” or “get 100% test coverage” rules are an answer in the long term as we haven’t yet figured out how to do this well.
I personally treat tests as components that need to be well architected, maintained, and add them risks increasing technical debt and needs to be treated as such.
Re:AI usage generates a fragile codebase
by elzbardico on Tuesday November 10, 2026 @06:50AM
(#50438994)
We shouldn't worry about that. We should be grateful about the mountains of technical debt, because that will be the social security of the post boomers generations: working till death fixing the last hurrah of the boomers.
Re:AI usage generates a fragile codebase
by retrac98 on Tuesday November 10, 2026 @06:51AM
(#50439005)
My experience has been sort of the opposite. In the past 6 months I don’t think I’ve introduced a single production bug through my work, which is unheard of (for me, at least).
LLM code is verbose, and often quite inflexible, but that rigidity and verbosity tends to come from the AI being very thorough about ensuring the code does exactly what’s expected, and it seems to leave much less room for error as a result. It acts as a really good forcing function for specificity on requirements, and for thinking through "what if" scenarios.
Re:AI usage generates a fragile codebase
by retrac98 on Tuesday November 10, 2026 @08:10AM
(#50439811)
In this instance, a monolithic rails API for a series of iOS apps with a few million daily active users. But I also work on some other TypeScript projects and have felt similar benefits there.
Re:AI usage generates a fragile codebase
by dreambigwrkhard on Tuesday November 10, 2026 @06:57AM
(#50439062)
The market will fix itself.
If the article is correct: AI generated codebase creates incidents that are more severe and take longer to recover from, companies will dial it back after they get burned bad enough.
If it's a skill issue: Maybe some coders can push the frontier out even a lot more than others before such things happen. Then the problem is the human in front of the screen.
Re:AI usage generates a fragile codebase
by fxwin on Tuesday November 10, 2026 @06:57AM
(#50439066)
> Here are the four most common ways that I have seen AI generating a fragile codebase. / Four signs that you have a fragile codebase
It would be nice if the listed symptoms (which have their own issues, e.g. "AI writes overly complex code" - overly complex for who?) were backed up with numbers or at least case studies, rather than stating "this is how you can recognize that X is happening", then stating "X is happening" in the title.
Re:AI usage generates a fragile codebase
by softwaredoug on Tuesday November 10, 2026 @07:00AM
(#50439087)
AI coding requires a lot of care to observability and verification.
If you can’t measure the problem, is it really a problem? A lot of assertions about code fragility seem to merge in with bike shedding arguments about style and what one person sees as readable.
But in the end if you have 100k of slop but we can know with high levels of certainty it does what it needs to - does it matter?
Every big company codebase with millions of lines devolves into a similar state. The small codebase starts with clean conceptual integrity. It then inevitably decays into incomprehensible spaghetti code as more people are added to it. Various reasons for it including the later devs caring less about the code than the initial team of devs, deadlines, avoiding refactoring because of fear about breaking things, etc
Re:AI usage generates a fragile codebase
by lofaszvanitt on Tuesday November 10, 2026 @07:10AM
(#50439184)
People the very least need to learn how to write memory prompts for their specific use case... Seemingly everyone is running with defaults or 100k+ stars skill abominations. My granma used this because important AI person said it is the way, look man it has 100k stars, it mussbegood!
If you feed back everything you had to fix with abominable intelligence generated code into your memory files, you'll end up with a pretty good prompt baseline that will steer the given model towards proper code generation.
Re:AI usage generates a fragile codebase
by thisisnotgreat on Tuesday November 10, 2026 @07:10AM
(#50439186)
> used javascript events for mouseover CSS effects rather than using the :hover selector, which is a totally unscalable way of writing mouseovers
It's good he gave an example because I frequently encounter these "fragility" arguments at work and it invariably turns out to be a giant F nothingburger. This was way before AI as well. Some type of people just cannot see the forest for the trees and this is a good example of that.
No mention of where this mouse handling took place but in my experience It Just Does Not Matter, but let's say it does. Let's say it's some super important bit of UX that suffers terribly from this approach.
I understand, I'm a dev. I'm familiar with how little things can matter. So sure, the mouseovers are implemented in a strange way. It causes issues in practice and not just in theory (which is IME 1% of cases), well, redo them. You think that takes more than an hour? If so, I question your ability to function in this job. Actually, document how mouseovers should be handled because apparently it's so important. This will never happen again. Think about the process, think about how you can prevent this. Don't focus on tiny little instances in time causes by tiny specific human beings, focus on the general, the universals. Everything you wrote shows me you are severely incapable of doing that. The fact you even focus on mouseovers and not on bigger issues shows me you have a questionable ability to allocate your scarce - it's scarce in all of us - cognitive capital.
Your fragility points all can be handled by skilled use of AI tooling. It's really F easy as well. "AI writes overly complex code", really? That's your point? Come on. Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.
In my practical experience your points are all false:
1) Our AI-driven products have less incidents
2) They are (way) easier to reason about
3) I don't even know what point 3 is going on about because devs famously only look at code and don't think about the end-user at all resulting in famous dev UIs
4) They all have a (much) better user experience, we're talking orders of magnitude like a different - more capable - company produced them
Sure, I hear ya'll geniuses shouting in unison our devs suck and you all could have done this in 5 seconds and are so, so superior, but guess what? Even if what you say is true, and it's not, you are few and far away and out of reach for us lowly regular businesses.
Re:AI usage generates a fragile codebase
by popkkksidhd on Tuesday November 10, 2026 @07:42AM
(#50439533)
> It causes issues in practice and not just in theory (which is IME 1% of cases), well, redo them
You often find issues “in practice” when your users find them. User trust is a limited resource that does not currently scale with LLM code generation rates.
> Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.
I (somewhat, it will choose its own path at odd times) agree, but the rate that it generates code dwarfs mine, so it’s generating issues “in practice” at levels I could never reach. AI + human is going to beat either in isolation in all cases, and I don’t see this ever changing until they can read our minds.
At my company, we have a “10x” AI team that doesn’t read the code. They had a lot of clout until they launched and were reprimanded for a staggering number of issues ranging from technical bugs to weird UX choices.
I’m not sure if their issues were caused by not reading the code or not knowing what they don’t know. Based on personal experience, if I stopped reading the code it would cause similar issues in my app, so I still read the code.
> hear ya'll geniuses shouting in unison our devs suck
The software world is huge. I’ve worked with devs who spent their whole careers in relational dbs and never heard of isolation levels. Entire modes of failure they don’t even know exist. I imagine they’d see code and claim it’s great where others would see the holes. And both could be right: there are business domains where it doesn’t actually matter.
The range of takes on LLMs is downstream of this diversity.
Re:AI usage generates a fragile codebase
by yodsanklai on Tuesday November 10, 2026 @07:11AM
(#50439197)
I don't think this matters anymore. Software is becoming a black box and agents make it works, as long as you can provide some reasonable spec of what you want to do, and a bit of supervising and directions.
Right now, they're not great at following good software engineering principles. They do the job, with verbose code, lots of duplication, abstraction leaks, useless unit tests and so on.
Is it an issue? you could think that in the long term it'll be, that AI won't be able to handle the complexity it introduced, but I'm not really worried about it. Most software isn't that big, and agents are getting better by the day. And if you really care about design, it's not too hard to instruct your agent to follow your standards.
I wish it wasn't the case and that I could justify my experience and expertise for a few more years, but the facts are there.
Re:AI usage generates a fragile codebase
by softwaredoug on Tuesday November 10, 2026 @07:54AM
(#50439659)
I have a Stackoverflow mode for my agents (essentially a tutor) and sometimes I spend time hand coding but heavily guided by an agent. It helps A LOT to make careful tradeoff when succinct and line by line care matters.
Re:AI usage generates a fragile codebase
by vkgcr on Tuesday November 10, 2026 @07:31AM
(#50439413)
So you're essentially willingly becoming 100% dependent on AI, while praying that hopefully they'll become good enough to save you from the slopocalypse down the line.
Re:AI usage generates a fragile codebase
by vkgcr on Tuesday November 10, 2026 @08:11AM
(#50439827)
I don't think there will ever be a way to "steer" an LLM such that it produces the correct output 100% of the time. You can certainly context engineer your way to better outputs, but you can never reduce to 0 the possibility of the LLM making a mistake, due to inherent limitations of the LLM (perhaps what you ask of it wasn't RL'd during its training), but especially due to the fact that it will often misunderstand requirements and make unjustified assumptions.
So, no, you will never be able to produce code automatically. You can and certainly should use AI as a code assistant to write code faster, but the human should always 1) make all important architectural decisions and 2) carefully review the outputted code
Re:AI usage generates a fragile codebase
by hoppp on Tuesday November 10, 2026 @08:17AM
(#50439887)
Not correct but you can plan in the mistakes.
E.g: I designed a modular monolith where each module is strongly decoupled so an LLM can work on it individually and if it breaks it can be ripped out and replaced without damage to the rest of the system.
So the system accounts for mistakes and has protocols for removing bad modules/code.
Re:AI usage generates a fragile codebase
by vkgcr on Tuesday November 10, 2026 @08:26AM
(#50439980)
Sure but that means accepting the cost of having faulty modules running in production (and there is no guarantee that your LLM could fix a faulty module, perhaps it will just replace it with a module with different bugs. There's no way to know!)
This obviously won't fly for "serious" software such as airplane software, but I'm willing to guess that even customers of a low-stakes SaaS would get pissed off if your product regularly turns up new bugs.
Re:AI usage generates a fragile codebase
by EPWN3D on Tuesday November 10, 2026 @09:31AM
(#50440930)
The difference with humans is that you can incentivize them to try and do the right thing via money. Humans are error-prone and slow, but they don't have an alignment problem, and the types of mistakes they'll make are pretty much understood. There are entire programming languages designed to minimize those types of mistakes.
The types of mistakes LLMs will
make are failures of alignment, that is, basically unpredictable. They could fuck up anything. Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.
It will happen less and less as they get better and better, but as the volume of software output keeps growing, it will happen with a consistent defect rate. And we won't be able to catch it prior to shipping, because humans won't be able to review all the code or verify all the tests.
I don't think anyone's started taking this seriously yet.
Re:AI usage generates a fragile codebase
by megak1d on Tuesday November 10, 2026 @07:35AM
(#50439449)
This is what I came to post.
For the last 20 years of my career, I have prided myself on keeping a clean codebase, making 'art-not-science' decisions of when and what to abstract, scope boundaries, then implementing them cleanly, specifically to be easily grok'd by other developers. But after building, from scratch, and operating an (essentially) vibe coded software solution (in typically a very engineer led/low level space) over the last 9 months, I am still able to move at super-human speeds in terms of delivering features, investigating and fixing any issues (there aren't many, certainly less than most human teams I've been in) - whether that's because of AI advances or just their innate quality of being able to tackle the dross coding tasks that always got avoided by humans in my past I'm not sure but I'll stick with it until a better approach becomes apparent!
I feel my specialist skills (coding, debugging, devops) has been devalued - I can't justify holding those as my value-add proposition any more - but new skills are replacing those (context/AI/agent engineering) in equal measure.
Re:AI usage generates a fragile codebase
by geertj on Tuesday November 10, 2026 @07:44AM
(#50439553)
I think I agree with all you said, but I can’t go of a clean codebase yet. And I’m not sure I should. Or at least not now, not until the agent can better understand what I actually want.
And you’re right, with a bit of direction and further prompting it’s quite easy to get your code in a reasonable shape.
Mostly my hope is that at some point soon the agents will start writing clean code. But then again, maybe clean code for a galaxy brained agent is not actually the most efficient code from token spend perspective.
Re:AI usage generates a fragile codebase
by mhitza on Tuesday November 10, 2026 @07:42AM
(#50439532)
Might be true in your company, your culture, or in Big Tech.
For a different perspective, the european peers in my developer network, while never outspoken against their company, like, and redistribute content that circle around the same points. Some have fixed mandates to use AI, in order to use AI, working more to keep the product stable (even outside normal work hours), or complains about the divide between management and developer's perspective on these tools.
There is also some consideration about compliance issues and liability when you're stamping commits with your name (luckily for almost all software enginners there is no malpraxis law to punish gross negligence).
Re:AI usage generates a fragile codebase
by yodsanklai on Tuesday November 10, 2026 @09:31AM
(#50440940)
> Might be true in your company, your culture, or in Big Tech.
For context, I work in a big tech company with virtually unlimited tokens on the best models, and a lot of tools integrated in our workflows. The culture makes it impossible to go against the wind. Even if you don't want to use AI as much as others, you won't be able to prevent them to do so.
Re:AI usage generates a fragile codebase
by zombot on Tuesday November 10, 2026 @07:44AM
(#50439560)
Meanwhile overlooking all the strategic implications of handing off your production to an external service provider, with built-in total surveillance and industrial espionage. All the software shops are turning into lemmings. Running faster into the abyss is called increased productivity. Full speed ahead!
Re:AI usage generates a fragile codebase
by softwaredoug on Tuesday November 10, 2026 @07:48AM
(#50439608)
The experience still matters.
I have had experiences out of my technical depth where I’ve instructed an agent to do something technically nonsensical. And it tried to do it or created a mess. Nothing works, I get frustrated, only to realize I have a fundamental misunderstanding of something technical
Re:AI usage generates a fragile codebase
by ffsm8 on Tuesday November 10, 2026 @07:54AM
(#50439658)
Fwiw, that's not what the parent was talking about.
You're trying to middle road it. He was talking about actually treating it as a black box, so just give the model your expectations and goals it needs to meet, not tell it a rough solution for it to apply in detail.
Re:AI usage generates a fragile codebase
by rsfern on Tuesday November 10, 2026 @08:43AM
(#50440231)
Expectations and goals can be technically nonsensical. Agents have gotten really good, but they can’t magically solve intractable problems. If you’re out of your technical depth it can be easy to set unrealistic expectations without realizing it
Re:AI usage generates a fragile codebase
by ffsm8 on Tuesday November 10, 2026 @08:56AM
(#50440418)
Then you're still middle-roading it from my perspective. Treating it as a black box means you're only specifying what comes out, essentially.
But to preempt a "I can't do that in my brownfield distributed monolith I'm maintaining at $dayjob": yes, that's why I said it may not be possible without refactoring.
Re:AI usage generates a fragile codebase
by rsfern on Tuesday November 10, 2026 @11:15AM
(#50442430)
I think we are talking past each other a bit? I’m curious about GP’s perspective here
My point is that you can ask for unrealistic outputs even if you’re treating the system as a black box. This is the classic case of higher level managers asking for infeasible results from their engineering team because they haven’t considered they’re trying to solve the wrong problem. Depends what you’re doing but I don’t think agents tend to push back on badly specified goals unless you’re proactive about it
Re:AI usage generates a fragile codebase
by utilize1808 on Tuesday November 10, 2026 @08:43AM
(#50440224)
That's right. I call it "organic software" --- software that behaves like living organisms --- lots of glitches and flaws but are constantly being fixed by AI agents. 99.8% of users never run into catastrophic issues, and for those who do, a dedicated human/agent will handle it manually.
Re:AI usage generates a fragile codebase
by fauigerzigerk on Tuesday November 10, 2026 @07:17AM
(#50439255)
I think shallow tests is the the most important issue on this list. AI works well if there's a good feedback loop. Without very good test cases and/or some kind of formal specification, the feedback loop is a haphazard, unrepeatable mess and code quality will inevitably decline.
Re:AI usage generates a fragile codebase
by sirius87 on Tuesday November 10, 2026 @07:30AM
(#50439400)
Has anyone figured out a good way to fix shallow tests with AI? Intuitively, in webapps, there are many minute UX things folks on the team used to manually dev-test. not saying that was done consistently across all team members or always done, but it existed.
Now it's gone. Spec-driven dev isn't the answer either, some of the UX bugs areas are simply not something that ever existed in the regression suite.
Re:AI usage generates a fragile codebase
by softwaredoug on Tuesday November 10, 2026 @07:41AM
(#50439522)
I spend most of my time structuring code to be meaningfully verifiable. To me that usually means turning a large functional area into a CLI. So for example I have a pipeline of processing and enriching products for search. Thats a CLI whereas before it might be a module in my larger codebase.
Then in that CLI I have a thin front end that just parses args and calls the backend that does the work.
I test around this backend. As a kind of behavioral test.
I rarely mock, and only at some external API boundaries. I may enumerate those (they’re only 2-3 things) the tests call my code via the backend, and I can verify more easily visually the test measures something meaningful. Moreover they map cleanly to the business logic of this module.
If testing becomes brittle I can converse with my agent and show it a bug via the CLI. It can repro more easily, and with a cleaner mapping between the CLI and my tests, I can confirm it tested what I expected.
Re:AI usage generates a fragile codebase
by sirius87 on Tuesday November 10, 2026 @07:55AM
(#50439668)
thanks. I guess the problem is more acute in larger codebases, where the code AI writes takes "inspiration" from existing human-written code (i.e. conforms to earlier human opinions), so it doesn't go out of its way to write meaningfully verifiable new code - because that would stick out like a sore-thumb in a code-review.
Writing modular, verifiable code was always an engineering tenet - but the use of AI in codebases that ignored those foundations ends up having several gaps.
Re:AI usage generates a fragile codebase
by sadilek on Tuesday November 10, 2026 @09:09AM
(#50440628)
For UX specifically, I have good experience with letting the agent record a demo video for me. It regularly finds issues itself while doing this and fixes them, and I get a chance to very efficiently see the UX w/o having to start a dev server and navigate somewhere and create a certain state before I can see the UX.
Re:AI usage generates a fragile codebase
by armanckeser on Tuesday November 10, 2026 @07:24AM
(#50439334)
A fragile codebase *today. What happens when the new generation models come out? Taking on debt today to move faster makes a lot more sense because the "interest rates", if you will, make the math such that new AI models can pay the debt their predecessor left. Bad tests were harmful because I had to hand fix them which meant spent time, today it just means 5 seconds for the AI model.
Re:AI usage generates a fragile codebase
by Nekorosu on Tuesday November 10, 2026 @07:25AM
(#50439341)
The article seems to repeatedly confuse product quality with code quality, as if they're interchangeable. They're not, and that confusion pretty much undermines the entire argument.
Re:AI usage generates a fragile codebase
by Nekorosu on Tuesday November 10, 2026 @10:22AM
(#50441666)
Code base is an emergent property of a product (that can have no code at all in practice). I'd even argue that the less code a product needs, the better — with no code being the ideal.
Re:AI usage generates a fragile codebase
by microsoftedging on Tuesday November 10, 2026 @07:33AM
(#50439435)
I agree with most of this, but the market will solve itself[0]. I do think the last analogy diminishes the point.
> Shopping for clothes from a website always results in worse fits than shopping in person.
Knowing how one brand's clothes fit means that this isn't always true, and you just adjust depending on the brand. I don't understand what is trying to be conveyed here.
Re:AI usage generates a fragile codebase
by lf88 on Tuesday November 10, 2026 @08:00AM
(#50439710)
AI in software development should be limited to security scans and maybe occasional bug-fixing.
We'll have the best of all worlds: more secure code bases, happier & not unemployed developers, human understanding of what the code does, and far less money for companies that are hijacking our future by developing a god-like artificial super-intelligence.
Re:AI usage generates a fragile codebase
by abalgir on Tuesday November 10, 2026 @10:09AM
(#50441478)
Vibe coding taught me the hard way. These principles confirmed it:
AI made it easier to create software. It didn't make production easy.
1 - Done does not mean DONE.
2 - The code gets over-engineered, and you will not know about it. Unless you have a large team reviewing the AI's work. Which defeats the whole purpose.
3 - AI reviewing the code can over-review. It chases unrealistic edge cases and kills your project.
4 - Maintainability is not native to AI. It builds code nobody can maintain later.
I built a machine around the agents that takes care of these issues, and I can safely say that vibe coding made me a better builder of solutions (not a better coder).
I say that as a frustrated slop fighter.
Is this not incentive enough to use AI more thoughtfully?
This would be at the level of helper functions < 10 lines of code.
That part was obvious, no need to explain it.
Every once in a while I go "manual". It's restful and entertaining to write syntax by hand. It has this peculiar dare I say autistic delight, sorting the colored blocks and getting a little kick out of it. I understand the nostalgia but the progress is breathtakingly slow - as expected. You cannot do 6502 assembly by hand and expect to be fast in any meaningful sense of the word.
I usually juggle four, five active projects at a time and make days worth of manual progress on them in a morning. Support calls and colleagues questions interject occasionally. It's much more work than before, but I like it because dev work is super boring and now it's getting more interesting.
Like many people try to say AI is raising the bar. What we call normal software now will be childrens' toys tomorrow. The world expects ambitious projects, stuff that is hard and stands out.
AI is great at following patterns and loves to write tests. Start with good patterns that most teams would not have the discipline to write by hand and it is possible to get very robust code.
Try:
What's useful is to have a second agent that reviews the tests impartially and trim the ones that are low signal.For our tests, we also added a small helper library[0] that cleans up tests to make them easier to read (smaller context for both humans and agents) as well as to write. Agents write average code by default. Your job as an engineer in this age is to help agents write good code.
[0] https://github.com/zeeq-ai/zeeq-biome
Tell me again where there is NO widespread automatic consensus about how imperative is that one HAVE TO use these things from a sandbox?
The last time this happened was when the whole world agreed that it is alright to take a device with multiple cameras, mics AND 24x7 internet connectivity to our most private places...
- they write shallow tests
- they write overly complex code
What do you do? You provide guidance, support and tooling so they can write better tests and code. It is the same with AI agents (plus the hassle of how to carry that knowledge around and transfer it). You are responsible.
You are also responsible for the understanding part: know what is being worked on, read/skim the code, inspect the outcomes, cross-check specifications, tests, product requirements. The 'moving too quickly' idea is the other side of the same coin, it is your responsibility to keep track of where the project is going. The same thing happens in a project where you have 10 developers (or 200), you cannot possibly read all of the code they write.
Seeing all of these as downsides is a choice.
When I do the same with an LLM, I'm just repeating the same guidance over and over again out of frustration just to get something done.
Comparing an LLM to a junior engineer is frankly a disgrace.
For that you need to actually read the code and it's somehow in odds with the whole vibecode idea.
Plus it won't give you the 10x boost the management expects. Only not looking at the code will.
It can only simulate the outcomes of intuition in its training set.
The likely uncloseable gap is that some intution is unspoken, literally felt by the brain's default mode network.
LLMs basically never shower, do the washing up, kick their foot in the dirt while sitting on a park bench with a coffee: the domain of the default mode network.
I'm sure the training data of LLMs contains a lot of those things, though.
But they are not the things themselves — they are incomplete representations; just words that approximately describe the application of that intution to the problem at hand.
Think about how you visualise complex code projects in your mind — how it's not quite diagrams or buildings or words but multiple things in a sort of multi-dimensional, itchy space that is satisfied when a really good implementation appears. Think about how often the code you have built retains that wordless shape long after you wrote it down. Think about how impossible it is to adequately describe it.
The downside: the model doesn't always have words to describe what it's feeling.
Those outcomes are more easily averaged and we are after all talking about artificial intelligence.
I just think it's important to note, every now and then, that words aren't feelings or thoughts, and that an author's next book cannot be calculated from only the words of their previous books, etc.
Compute > Insight. Every time.
We can keep searching for essential human characteristics that humans have and AI lacks, but every attempt increasingly resembles an ego-preserving negotiation with the inevitable march. It's reminiscent of the God of the Gaps[2], but for human exceptionalism. As AI competency encroaches on more and more of human ability, we'll retreat into smaller and smaller gaps - foxholes of inimitablity.
As long as human value is justified by ability, quality, or characteristic, this problem will endure. The one escape hatch is the classical liberal idea that human value isn't tied to anything, that, by virtue of being human, people have universal, inalienable worth.
We should stop the project to attempting to substantiate human value by other means.
1. https://en.wikipedia.org/wiki/Bitter_lesson
2. https://en.wikipedia.org/wiki/God_of_the_gaps
I think speed is the biggest flaw though. If i could get variable speed while planning vs building, thatd change some things. I run local models, so i am mostly in the slow lane. When patching these flaws, i want it to read the code parts and diagnose quickly.
When its coding, i want to see the problems its encountering with the prompt. I usually observe its thinking a few rounds and if it misunderstands, i redirect.
But in all of this, i want a dial that just speeds up once i feel its doing the right thing.
SOTA today, to me, would put a model floor like qwen3.8-flash-next on a platform i can dial up the speed. Thats all. I think many of us will bring our own harness, so the only hard point is how dumb the engineers tools are.
Since im building local, my tools do a lot more management. So.e of tge opinions i see on HN trigger a "skill issue" reflex because we have a whole shopping mall now if tools and the incurious seem oblivious to how these models are used by cloud providers.
Learning about the raw capabilities of LLMs makes getting them to "do the right thing" is now just a job description.
Lastly, the last decade of tech oligarchy exposes how little education elites had about humanity and wicked problems. Elon musk reinventing public transportation several times ignorantly. The fact that a social program requires the consent of the people it helps. LLMs are a stark lesson in the difficulty of knowing all the constituent parts and still being unable to predict outcomes.
Welcome to Socioligy 101 tech nerds.
In interviews, he admitted that the Hyperloop was intended to mess up the California High Speed Rail project because trains are boring. It's the opposite of ignorance, it's malice. Hanlon's inversion.
On code readability, shallow tests and general quality, I can agree emphatically, though not universally. In other words, there are models which write the densest, hardest (for me) to parse code and tests that break with the lowest load/timing deviations yet proof less than nothing.
These do that regardless of how much one ships in my experience however so I wouldn’t attribute this to the velocity and more poor data at the heart.
I know this reads a bit like “you are using the wrong model” and I understand if someone testing e.g. an OpenAI models understandably extrapolates across all models and considers poor parse-ability in code a likely result of velocity, but you can always step back and work slower, plus, for what it’s worth, I struggle to parse OpenAI code when running their models through my evals after step 7 which is still a fairly measured in scope code base.
Essentially, I have seen LLMs, even the newest frontier, produce barely legible slop when building simple CRUD and I have seen a massive rewrite of the existing Firefox code base using APZ to enable an infinite canvas of browser tiles grow in a manner where, even at this stage having replaced a lot outside Places and Gecko, I can walk through changes made and explain the logic, as well as look at any part I haven’t been in for a while and find my bearing even without assistance.
AI is a force multiplier in whatever you apply it to. Most software companies output binaries with new features, whether it runs locally or on a server. Binaries with new features. None of the business owners give a shit about what is in the binary. It may be extremely fragile and only work for one type of user, but if they only have one type of user nobody cares. Most software doesn't keep evolving forever and at some point gets rewritten as a 2.0 version or whatever, so the quality was never as important.
It doesn't have to be that way. You can use AI just as effectively to produce quality source code as you can slop binaries. It depends on the business priorities and what is actually important.
The caveat is if you are the one engineer on the team trying to use AI the right way to produce quality source instead of binaries as fast as possible, you might get left behind or let go since your work no longer aligns with the business.
Disclaimer: I have no SaaS platform, it's a mobile app.
In my experience maintenance does not suffer long term with codebase size, if anything it became easier with model advances.
Most people talking about suboptimal implementation, fragility, and maintenance burden seem to theorize and extrapolate with no actual experience in implementing a large project with AI.
I can't blame them, I see the same in my day job, but the results after 500h+ on a separate project show that the concerns do not manifest in practice.
Also, sharing a link would be great if you're comfortable with the idea.
I'm still in the process of launching the app, it went live just two weeks ago. So far only 30 users, I have not yet started the ad campaign.
Basically I don't look at the codebase at all, I have long stopped using the IDE or doing code reviews.
From a product perspective, there have been no big surprises. Only once I was surprised to see a feature I thought I implemented only on a specific screen being available on a different screen too, hidden behind a drag gesture, spawning an UI component I totally did not expect.
While that seems like a loss of control, it's not too different from enterprise work with many contributors where I am also not aware of all changes.
About maintenance cost, no, I do not think so. It generally became easier over time since Opus 5.5 delivers better results than GPT-5.5 did.
From a token perspective, a large part of the resources go into translating to 30 languages and checking the screenshots for layout regressions.
Sorry, but at this point you have no idea what the maintenance looks like.
I've heard this argument a few times on here, I think it's nonsense.
At 30 users you have no idea what maintenance looks like. That's literally what maintenance is - responding to real world usage.
Come back with your experience when you have lots of users that stick with your app, you're putting out fires and implementing new features. Your experience will be meaningful after 6 months to a year.
I'll wager that right now you have a PoC that needs a lot of work to be something that retains users. That's not a criticism, that's just how it works for every startup.
I have a pretty good idea about what maintenance looks like. Particularly so about the technical aspects of how effective the models are at refactoring or fixing bugs.
Even if I were to accept the argument that users would have entirely different expectations for my app than I do, it hardly changes the fact that I already know I can easily implement large changes in the code base.
Come back after running your mobile app for at least 6 months in the real world with real users and then we will talk about maintenance.
Codebase has never been better and healthier, and any tech debt that has been accumulated after a few releases can be removed in a few prompts when you are out of product ideas and have some free tokens to spare.
(I added US mountains and trails btw: https://fescims.com )
<Error> <Code>NoSuchKey</Code> <Message>The specified key does not exist.</Message> <Key> releases/e17a06bd04efe2b18bb38c0ae905a4d7db7ac863-42361807812-1/en/challenges.html </Key> <RequestId>7RNT6H476HS6B22H</RequestId> <HostId> YZf8hkSztCkLmtCZ01b7L4ufaujln4rD7qmakrvPWcIqTpmUwexzkW9UJWta5YJnszCS7qkdGmPFyfC9na3L6As7skPPnu2f </HostId> </Error>
Yes, a modern model will happily work on the most unmaintainable slop (doesn’t even need to be LLM derived slop, we have all seen bad Human written code) and that is great if you have an old code base to maintain. But if one can avoid it, even if one uses only models and never looks at the code, being efficient has value. Heck, rewrites are doable and affordable now, I have restarted projects and the gains were quickly apparent, even if we just look at model usage efficiency gains.
Even if I ended up with a project of 50k lines, that would take multiple times the effort it took me to get this app ready for the initial release.
Realistically, I wouldn't have finished the project at all.
It doesn't happen often to me. If it does, I find it often handles extra edge cases I only would have found by using the customer as a guinea pig.
Where else did you think they came up with the "10x developer" thing?
(Yes I know LoC is a useless productivity measure, that's just a joke. Obviously it's number of PRs..)
But in doing so, you will also 10x the features delivered. Maybe 5x, who knows?
But you definitely won't have to go through and decimate the generated code as a rule.
Let me tell you the story of the developer trying to reduce the number of tokens they use...
At the current pace, the very idea of maintaining software will become as relevant as the idea of trimming wicks, stropping razors - or making every frame you take with a camera count.
When the marginal cost of replacement drops to a sufficiently low level, the entire paradigm changes.
Digital cameras are another interesting one. Since capturing images is so cheap we just snap a bunch of frames without a lot of thought towards composing them carefully, with a side effect that we get an overwhelming amount of mediocre frames and a few good ones, but we have to sift through.
I’m still not convinced I want to make this tradeoff in programming. But I work in scientific computing where I think it’s (still) really important to understand what modeling assumptions you are making through code
This reminds me of Moore's law in the 1990s.
If you program run slowly, or used too much RAM, you could spend six months optimising it (or removing features). Or you could just sit on it for six months and release it as it is - hardware moved on so fast that, by then, it probably ran successfully.
Your suggestion is a bit like this: don't worry, the pace of model understanding will catch up faster than the code will get disorganised.
I'm not saying I agree but it's an interesting idea.
My code has never been stronger.
Not saying it's better or worse. But it's definitely different.
My first startup was building requirements management software for small businesses to solve that exact problem. Devs add things in all the time. It's usually referred to as 'gold plating' or scope creep.
The belief that AI can do all that without the gates is throwing away something useful where people get to understand how the system is changing. If you keep the gates but use AI to generate all the code you either slow AI down to the speed of humans or you end up with a massive backlog of stuff to review.
This is a hard problem.
They aren't going to be creating 100 liabilities/hour?[1]
--------------------
[1] Each line of code added is a liability. We appear to have forgotten that.
Another of the shallow arguments thrown around, similar to "we don't check compiler output so..."
People learn and improve. LLMs don't learn, they follow instructions to a degree.
A human has to wrestle with it to work how you want, ergo humans are necessary already.
You can't fire the LLM and hire a competent one. The few available is all there is, they're all copies of the same thing.
You don't have a team. If you think 10 LLMs = a team of 10 people, then you don't know what a team is, how different skills and views make a big difference to outcomes.
LLMs can't innovate. They can't come to you with a great idea they came up with. They don't have ideas.
If none of these things are worth anything in your particular case, then that's ok. But let's not actually lose our minds and forget why people are intrinsically better than machines.
There are only a few arguments to support using LLMs, all of which are from people who a) want to spend less on salaries or b) sell you LLMs.
Teams are often hierarchical where the leaders define everything and people under them have less things to say about it. Not much different than an LLM team.
In general any type system is designed to prove things. So all I'm really saying is "use strong types to prove more" which is irrefutable.
just curious, and tempted to do the same, so far I don't like the code made by llms in dynamic languages codebases
Type systems are not a panacea. You still have to be logical and careful about them, especially around boundaries where types are just assumptions and not logical inferences. Then you have functions that switch type, but only a subset of the input is actually valid (string -> int for a number parser). Checking for NaN is like checking for Null.
Type systems does help, but they need the same attention as code to be actually helpful.
Step 3: Prompt the agent to write mutation tests to harden the suite
Step 4: Prompt the agent to write property-based tests.
Step 5: Prompt the agent for 100% code coverage
Step 6: Keep a look out for small issues with steps 2-5 above and refine a Skill or Rule to package the workflow and plug gaps in the model's current shortcomings.
Step 7: Dedicate some time to manually reviewing and thinking about interesting edge cases worth testing.
---
If you follow these steps, you will have better test coverage than 99.9% of any code an engineer ever wrote pre-AI... and it will be almost completely automated.
My team's incidents and bugs are DOWN this year while we're shipping more.
Leveraging AI effectively requires a new paradigm of interaction. The name of the game is to use techniques and strategies and guardrails that create an environment that produce good software.
I don’t think anyone has figured out the proper solution yet, I personally try to get the LLM to write code similar as I would have written it myself.
If you mean asserting against hardcoded literals, then that is the preferred testing shape in most cases. And for good reason.
If you change production code and a unit test fails, that is useful. Idk why that's silly. If you want to test more complex interaction write a UITest or integration test.
I honestly think we're all living in different worlds. There's probably large differences in capability on a per-language, per-team, per-codebase, per-engineer environment that are producing wildly different perceptions in output quality.
And yes everyone works in different domains. I work at a database vendor myself, so mostly lower-level work.
I’m very happy and successful with LLMs, I just don’t think that these “just move all your tests to your type system!” or “get 100% test coverage” rules are an answer in the long term as we haven’t yet figured out how to do this well.
I personally treat tests as components that need to be well architected, maintained, and add them risks increasing technical debt and needs to be treated as such.
I was describing general strategy, that when applied in concert is more than the sum of its parts.
E.g I mentioned property-based testing, that is another angle at which a test avoids assertions that are too fine grained.
I think it has much more to do with overall architecture and design, and as for now, human oversight.
LLM code is verbose, and often quite inflexible, but that rigidity and verbosity tends to come from the AI being very thorough about ensuring the code does exactly what’s expected, and it seems to leave much less room for error as a result. It acts as a really good forcing function for specificity on requirements, and for thinking through "what if" scenarios.
If the article is correct: AI generated codebase creates incidents that are more severe and take longer to recover from, companies will dial it back after they get burned bad enough.
If it's a skill issue: Maybe some coders can push the frontier out even a lot more than others before such things happen. Then the problem is the human in front of the screen.
It would be nice if the listed symptoms (which have their own issues, e.g. "AI writes overly complex code" - overly complex for who?) were backed up with numbers or at least case studies, rather than stating "this is how you can recognize that X is happening", then stating "X is happening" in the title.
If you can’t measure the problem, is it really a problem? A lot of assertions about code fragility seem to merge in with bike shedding arguments about style and what one person sees as readable.
But in the end if you have 100k of slop but we can know with high levels of certainty it does what it needs to - does it matter?
The fact that your bad engineers are now able to be bad 100x faster just makes it more of a disaster that you've got bad engineers.
Quit blaming AI and focus on adjusting your salaries and hiring practices.
Here's the famous commentary about Oracle RDBMS codebase of 25 million lines: https://news.ycombinator.com/item?id=18442941
Every big company codebase with millions of lines devolves into a similar state. The small codebase starts with clean conceptual integrity. It then inevitably decays into incomprehensible spaghetti code as more people are added to it. Various reasons for it including the later devs caring less about the code than the initial team of devs, deadlines, avoiding refactoring because of fear about breaking things, etc
If you feed back everything you had to fix with abominable intelligence generated code into your memory files, you'll end up with a pretty good prompt baseline that will steer the given model towards proper code generation.
It's good he gave an example because I frequently encounter these "fragility" arguments at work and it invariably turns out to be a giant F nothingburger. This was way before AI as well. Some type of people just cannot see the forest for the trees and this is a good example of that.
No mention of where this mouse handling took place but in my experience It Just Does Not Matter, but let's say it does. Let's say it's some super important bit of UX that suffers terribly from this approach.
I understand, I'm a dev. I'm familiar with how little things can matter. So sure, the mouseovers are implemented in a strange way. It causes issues in practice and not just in theory (which is IME 1% of cases), well, redo them. You think that takes more than an hour? If so, I question your ability to function in this job. Actually, document how mouseovers should be handled because apparently it's so important. This will never happen again. Think about the process, think about how you can prevent this. Don't focus on tiny little instances in time causes by tiny specific human beings, focus on the general, the universals. Everything you wrote shows me you are severely incapable of doing that. The fact you even focus on mouseovers and not on bigger issues shows me you have a questionable ability to allocate your scarce - it's scarce in all of us - cognitive capital.
Your fragility points all can be handled by skilled use of AI tooling. It's really F easy as well. "AI writes overly complex code", really? That's your point? Come on. Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.
In my practical experience your points are all false:
1) Our AI-driven products have less incidents 2) They are (way) easier to reason about 3) I don't even know what point 3 is going on about because devs famously only look at code and don't think about the end-user at all resulting in famous dev UIs 4) They all have a (much) better user experience, we're talking orders of magnitude like a different - more capable - company produced them
Sure, I hear ya'll geniuses shouting in unison our devs suck and you all could have done this in 5 seconds and are so, so superior, but guess what? Even if what you say is true, and it's not, you are few and far away and out of reach for us lowly regular businesses.
You often find issues “in practice” when your users find them. User trust is a limited resource that does not currently scale with LLM code generation rates.
> Tell it not to. Instruct it, shape the process. Define your terms, document your ideal code style. I guarantee you it's better at it than you are once it knows what you want.
I (somewhat, it will choose its own path at odd times) agree, but the rate that it generates code dwarfs mine, so it’s generating issues “in practice” at levels I could never reach. AI + human is going to beat either in isolation in all cases, and I don’t see this ever changing until they can read our minds.
At my company, we have a “10x” AI team that doesn’t read the code. They had a lot of clout until they launched and were reprimanded for a staggering number of issues ranging from technical bugs to weird UX choices.
I’m not sure if their issues were caused by not reading the code or not knowing what they don’t know. Based on personal experience, if I stopped reading the code it would cause similar issues in my app, so I still read the code.
> hear ya'll geniuses shouting in unison our devs suck
The software world is huge. I’ve worked with devs who spent their whole careers in relational dbs and never heard of isolation levels. Entire modes of failure they don’t even know exist. I imagine they’d see code and claim it’s great where others would see the holes. And both could be right: there are business domains where it doesn’t actually matter.
The range of takes on LLMs is downstream of this diversity.
Right now, they're not great at following good software engineering principles. They do the job, with verbose code, lots of duplication, abstraction leaks, useless unit tests and so on.
Is it an issue? you could think that in the long term it'll be, that AI won't be able to handle the complexity it introduced, but I'm not really worried about it. Most software isn't that big, and agents are getting better by the day. And if you really care about design, it's not too hard to instruct your agent to follow your standards.
I wish it wasn't the case and that I could justify my experience and expertise for a few more years, but the facts are there.
Depends on use case
Doesn't seem like a good strategy to me.
Using AI is one element for a pipeline that produces code.
If you architect well what you generate then it stops being slop and starts behaving like a cohesive system where code is produced automatically.
So, no, you will never be able to produce code automatically. You can and certainly should use AI as a code assistant to write code faster, but the human should always 1) make all important architectural decisions and 2) carefully review the outputted code
The human bottleneck is necessary.
So the system accounts for mistakes and has protocols for removing bad modules/code.
This obviously won't fly for "serious" software such as airplane software, but I'm willing to guess that even customers of a low-stakes SaaS would get pissed off if your product regularly turns up new bugs.
I think it's easier to just do code reviews...
Both me and LLMs are constantly auditing software.
In-fact auditing and QA is mainly what I do now and human audits become much easier when it's scoped to modules.
You cannot do that with humans either.
The types of mistakes LLMs will make are failures of alignment, that is, basically unpredictable. They could fuck up anything. Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.
It will happen less and less as they get better and better, but as the volume of software output keeps growing, it will happen with a consistent defect rate. And we won't be able to catch it prior to shipping, because humans won't be able to review all the code or verify all the tests.
I don't think anyone's started taking this seriously yet.
Gen Z has learned that usually, your dedication to work is not tied to the monetary reward.
> Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.
Have you ever done a group work project at school?
For the last 20 years of my career, I have prided myself on keeping a clean codebase, making 'art-not-science' decisions of when and what to abstract, scope boundaries, then implementing them cleanly, specifically to be easily grok'd by other developers. But after building, from scratch, and operating an (essentially) vibe coded software solution (in typically a very engineer led/low level space) over the last 9 months, I am still able to move at super-human speeds in terms of delivering features, investigating and fixing any issues (there aren't many, certainly less than most human teams I've been in) - whether that's because of AI advances or just their innate quality of being able to tackle the dross coding tasks that always got avoided by humans in my past I'm not sure but I'll stick with it until a better approach becomes apparent!
I feel my specialist skills (coding, debugging, devops) has been devalued - I can't justify holding those as my value-add proposition any more - but new skills are replacing those (context/AI/agent engineering) in equal measure.
And you’re right, with a bit of direction and further prompting it’s quite easy to get your code in a reasonable shape.
Mostly my hope is that at some point soon the agents will start writing clean code. But then again, maybe clean code for a galaxy brained agent is not actually the most efficient code from token spend perspective.
For a different perspective, the european peers in my developer network, while never outspoken against their company, like, and redistribute content that circle around the same points. Some have fixed mandates to use AI, in order to use AI, working more to keep the product stable (even outside normal work hours), or complains about the divide between management and developer's perspective on these tools.
There is also some consideration about compliance issues and liability when you're stamping commits with your name (luckily for almost all software enginners there is no malpraxis law to punish gross negligence).
For context, I work in a big tech company with virtually unlimited tokens on the best models, and a lot of tools integrated in our workflows. The culture makes it impossible to go against the wind. Even if you don't want to use AI as much as others, you won't be able to prevent them to do so.
I have had experiences out of my technical depth where I’ve instructed an agent to do something technically nonsensical. And it tried to do it or created a mess. Nothing works, I get frustrated, only to realize I have a fundamental misunderstanding of something technical
You're trying to middle road it. He was talking about actually treating it as a black box, so just give the model your expectations and goals it needs to meet, not tell it a rough solution for it to apply in detail.
But to preempt a "I can't do that in my brownfield distributed monolith I'm maintaining at $dayjob": yes, that's why I said it may not be possible without refactoring.
My point is that you can ask for unrealistic outputs even if you’re treating the system as a black box. This is the classic case of higher level managers asking for infeasible results from their engineering team because they haven’t considered they’re trying to solve the wrong problem. Depends what you’re doing but I don’t think agents tend to push back on badly specified goals unless you’re proactive about it
This raises alarm bells. It's a deeply unserious take on software development that I hope doesn't spread (but I know will). YOLO it all.
We absolutely will regret the slop AI, as it will become load-bearing (ahem) in short order. There will be critical outages and people will die.
Now it's gone. Spec-driven dev isn't the answer either, some of the UX bugs areas are simply not something that ever existed in the regression suite.
Then in that CLI I have a thin front end that just parses args and calls the backend that does the work.
I test around this backend. As a kind of behavioral test.
I rarely mock, and only at some external API boundaries. I may enumerate those (they’re only 2-3 things) the tests call my code via the backend, and I can verify more easily visually the test measures something meaningful. Moreover they map cleanly to the business logic of this module.
If testing becomes brittle I can converse with my agent and show it a bug via the CLI. It can repro more easily, and with a cleaner mapping between the CLI and my tests, I can confirm it tested what I expected.
Writing modular, verifiable code was always an engineering tenet - but the use of AI in codebases that ignored those foundations ends up having several gaps.
> Shopping for clothes from a website always results in worse fits than shopping in person.
Knowing how one brand's clothes fit means that this isn't always true, and you just adjust depending on the brand. I don't understand what is trying to be conveyed here.
[0] https://news.ycombinator.com/item?id=50439062
Every six months, the number of people capable of writing code better than AI is halved! :D
1 - Done does not mean DONE. 2 - The code gets over-engineered, and you will not know about it. Unless you have a large team reviewing the AI's work. Which defeats the whole purpose. 3 - AI reviewing the code can over-review. It chases unrealistic edge cases and kills your project. 4 - Maintainability is not native to AI. It builds code nobody can maintain later.
I built a machine around the agents that takes care of these issues, and I can safely say that vibe coding made me a better builder of solutions (not a better coder).