Wow, it’s true, AI really is set to match human performance on large, complex software systems! ;)
Including all of the above.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
A decade ago, I was sitting in on a meeting about a rewrite and, before I could say anything, someone in the first year of her career asked why anyone thought a rewrite would be any cleaner once all the edge cases were handled. Afterwards, I asked her where she learned this. She said "I don't know, it just seems kind of obvious." She went on to be a great engineer and is now a great manager.
Greenfield guy comes in, promises the world, and starts from some first principles white papered architecture. It's really lovely until they onboard the first user. Then they slowly commit all the "sins" (features that drive revenue) of the first system.
The firm is stuck supporting N systems indefinitely because the perfect new system takes so long to cover even 30% of the original system use cases, that management takes a flier on.. bear with me.. a second rewrite. Now they have 3 systems.
I've seen more 3rd systems than I've seen actual decommissioning of original systems into a single clean new system.
The answer is chipping away, modularizing, and replacing piecemeal Ship of Theseus style. But that does not drive big hires and big promotions.
Do they??
There is a lot of absurdly complex software that runs with high reliability. We hear a lot about the ones that don’t.
My team lead has worked on the same software for 30 years. He has the ability to hear me discuss a bug I noticed, and then pinpoint not only the likely culprit, but the exact function that's causing it.
And with one you need to train a guy for 25 years and with the other you need plan mode for a few minutes and then it runs 24/7.
And the equivalent for software. It’s usable, intuitive, responsive, stats up and running, and doesn’t leak my private data.
Then the only "experts" (not even close, just a guy with a form and some technical training) are the building inspectors who come at the end to verify if some stuff is done up to code.
Other than the original architect who draw the plans that got used for many buildings and the electrical engineer that cleared the electrical, no experts were involved. This is basically how the whole city and most of the country was built.
There's no expert mason or painter or whatever involved. Just a dude that can hold a paint roller. That's the same as going from a craftsman programmer to some dude with claude. Individual quality goes down, but more importantly price goes down way more and so many more people get access to much better quality than having nothing.
Younger implies cheaper.
I have really tried as an "old" person in the field to try and pass on the stuff I've learned, but "craft" and such really has absolutely no home in modern dev culture. The people who care about history, the craft, etc. are increasingly rare.
maybe some that people said were that bad. but they just needed some elbow grease. remember, it takes guts to be amazing!
I think it will be needless verbose complexity.
I kind of imagine someone having an unlimited budget of free amazon stuff shipped to their house.
In theory, they are living a prosperous life of plenty.
In reality, they will be drowning in something that isn't prosperity.
The explanation, in turn, can be fed back to recreate the functionality of the original code.
At that point, why care about the code at all? If it works, it works. If it doesn't, tell the model to fix it. You did ask for tests, right?
That is where we're indisputably headed. It's not quite a lossless loop yet, but those who say it won't or can't happen bear a heavy burden of proof.
On one end, you have code that can perform only the behaviour explicitly declared in the spec, but has to be thrown away and rewritten for any new or updated spec.
On the other end, you have code that implements or anticipates a wide range of future possible specs including the given one.
The AI can operate on any point on this spectrum, but it's not very good at choosing. The more complex the software, the more such choices need to be made.
When the number of bad choices reaches a certain critical mass, even a skilled engineer becomes powerless to undo all the bad choices, and even a powerful model becomes unable to reduce it back to a coherent spec.
It is now, and vice versa. Deal with it.
Some people are mindful about what they get and don't get from amazon and don't die from prosperity. ("you might use AI to increase your prosperity")
the rest of the world eats too much and dies of heart disease/diabetes. ("the rest of the world will flounder more and AI will do more stuff to them than for them")
The reason Oracle can continue failing at those massive projects is simple: everyone fails at them routinely and often it’s the customers fault.
It's just an umbrella term for "weak process glue code".
It's even simpler. Youre not paying oracle for some delapidated HR system. You're paying for the legion of accountability that is their on-site engineers to fix stuff for you when things screw up. You're essentially subscribing to a team of engineers you don't need to directly pay salary and benefits to.
People who think you can out efficiency that kind of accountability don't understand how large orgs think.
it will kill all the people in that hospital too
> On January 3, 2022, the jury found Holmes guilty on four of the seven counts related to defrauding investors: three counts of wire fraud, and one of conspiracy to commit wire fraud. She was found not guilty on four counts related to defrauding patients
What do you think the fake Delve attestation scandal was about? https://news.ycombinator.com/item?id=47444319
(Screams in "deployed in 2026 a new product that only works in internet explorer" in healthcare).
That sadly does seem to be the trajectory of 5-10 years from now, though. I can't speak to if "AI is the future" of 30+ years from now, but these coming years sounds rife for "janitors" to clean up all the slop being produced by newly empowered idea guys
Definitely cleaning up other people's AI mess for them for free is not a good use of time.
People really have a misconception about the sums of money that companies operate on on a regular basis. If you are a people person and know essentially how to sell yourself, you can "scrape" money on the fact that nobody is going to look or think too hard about some contract that represents a tiny fraction of the years budget.
Dear Lord. Respect to your friend for mad marketing skills, however. Selling slop to mission-critical sectors is next level.
“ These are highly complicated pieces of equipment… almost as complicated as living organisms.
In some cases, they’ve been designed by other computers.
We don’t know exactly how they work.”
Now how did that work out ;-)
I think the problem will get worst. I dislike the marketing around AI, but I do think it is a useful tool to help those who have experience move faster. If you are not an expert, AI seems to create a complex solution to whatever it is you were trying to do.
I've been watching non-developers vibe code stuff, and the general failure mode seems to be ignorance of 3-pick-2 tradeoffs.
They'll spam "make it more reliable" or some such, and AI will best-effort add more intermediary redis caches or similar patterns.
But because the vibe coders don't actually know what a redis cache is or how it works, they'll never make the architectural trade-offs to truly fix things.
I often wonder if it’s the statistical nature of the LLM mixed with a request in the prompt.
Here’s a slightly different future - these AI rescue consultants are bots too, just trained for this purpose.
Plausible?
I have already experienced claude 4.7 handle pretty complex refactors without issues. Scale and correctness aren’t even 1% of the issue it was last year. You just have to get the high level design right, or explicitly ask it critique your design before building it.
Do you think people are not giving their agents specs and asking for input?
Doesn’t work well ofc in a one shot situation with no context.
Commits, design reviews, whitepapers, code reviews, test suites. And pretty concerning---ai chat logs and even keystrokes from employees nowadays.
The way we train specialized bots now is incredibly inefficient, that part is rapidly improving.
That's serious levels of circular thinking right there.
We train humans to do things untrained humans can not do.
That's not at all how AI training works.
- AI Hype
- AI Psychosis
- AI keeps getting better and better until it can work around big AI slop code bases
The belief in this is a form of AI psychosis, I think.
Maybe in the future but certainly no evidence of this anytime soon
Here's some anecdotal evidence from me - I cleaned up multiple GPT 4.x era vibecoded projects recently with the latest claude model and integrated one of those into a fairly large open source codebase.
This is something AI completely failed at last year.
Maybe you should try something like this or listen to success stories before claiming 'certainly no evidence' in future?
What evidence is there that we're not at or close to a plateau of what LLMs are capable of? How do you know the growth rate from 2023 to present will continue into 2029? eg. Is it more training data? More GPUs? What if we're kind of reaching the limits of those things already?
And the answer appears to be that the improvement is accelerating. So how could it be stopping?
I don’t think that the current AI paradigm has infinite headroom for improvement, similar to how every other AI approach before it eventually hit a limit.
And the link I posted shows the amount of work a query can do increasing non linearly. You can explore the site for more detail and a graph that shows error rates getting halved every couple of months.
No one said anything about infinite. It doesn't mean we don't have headroom to spare.
Software itself took 80-120 years to get where it is today depending on how you count. Time is on AIs side here.
I don't see why we would assume that we are at a plateau for RL. In many other settings, Go for instance, RL continues to scale until you reach compute limits. Some things are more easily RL'd than others, but ultimately this largely unlocks data. We are not yet compute/energy/physical world constrained. I think you would start observing clear changes in the world around you before that becomes a true bottleneck. Regardless, currently the vast majority of compute is used for inference not training so the compute overhang is large.
Assuming that we plateau at {insert current moment} seems wishful and I've already had this conversation any number of times on this exact forum at every level of capability [3.5, 4, o1, o3, 4.6/5.5, mythos] from Nov 2022 onwards.
The (leading) LLMs work by consensus, like Wikipedia, Openstreetmap, web search engine or opensource movement.
What I mean is if I ask LLM "create a linked list", its understanding (of what I want) is already close to the expected ideal. Just like Wikipedia article on linked list, for example.
But the LLMs will continue to improve in breath and depth of understanding the world, although technically (what they CAN do) they probably already peaked. Similarly, OSS movement technically peaked in the 90s with the creation of compiler, operating system and a database; doesn't mean that new opensource isn't being created.
LLMs (or specifically GPT algorithm) are 8 years old. It has matured as a technology. I am not sure how you imagine it being significantly improved, from a user point of view, without some kind of paradigm shift (i.e. something significantly different from GPT or LLM).
Although I can imagine one important social innovation yet to come - a generally available big public LLM, that "anybody can train". We had a technology of "encyclopedia" for years (famously Brittanica); yet the concept of Wikipedia has been a truly new take on encyclopedia.
Also, new kinds of AI might emerge - for example we might formalize all types of human reasoning and build a reasoning AI, as well a model of human language, from scratch rather by training via GPT (and thus, more understandable and potentially smaller). But that won't be an LLM.
I proposed how. New harness techniques and new training data/techniques, so the harness gets better and the LLM can be trained to work better with the harness. There's no reason to believe we're out of momentum for improvement in that direction.
However, they also make mistakes like humans, I don't think a better harness or better training will fix that, because fundamentally, they cannot read your mind, if you put in an ambiguous prompt.
I like to compare the process of turning inexact text to formal language to an error-correcting code. If you haven't made too much mistakes or have been precise in the specification, it will self-correct and do what you want. But if your input is too ambiguous, it will never do exactly what you want, but something close to it. And people (who are using AI) are still learning where is the boundary and how to tell.
The companies building these models are training them to react to typical expectations. If you have some special need, you will always have to tell the model, otherwise it will not know your exact context. And the harnesses have many tools for that or try to do that automatically already.
* A belief that AI will keep getting better, presented without evidence, does not yield a lot of skepticism around these parts.
* Your comment saying it is wrong to believe AI will keep getting better, also presented without evidence, is downvoted.
1) same business logic implemented in two different places, with extra code to sync between them
2) fixing apparently simple bugs results in lots of new code being written
It’s a sign I need to at least temporarily dedicate more effort to overseeing work in that area.
I somewhat agree with the AI psychosis framing of the OP. It takes some taste and discipline to avoid letting things dissolve into complete slop.
I don't know what happens in a decade when there are no junior engineers, skilled senior engineers are becoming rare, and the only data left the train LLMs on is 200th-generation slop. But AI slop being qualitatively slop is not enough of a obstacle to prevent that future from coming to pass. And billions of dollars will be "saved" along the way.
That's what makes this whole house of cards dangerous. The prescription to psychosis is profitable. Aka, selling a grift.
I instructed it to split it up anyway, yet I wonder how often the concerns around the mess are imaginative rather than practical.
You have not seen the spreadsheets that accounts run the firm on.
Bloody kids!
I exaggerate only a little.
Are you sure about this? Yes, there is a stable set, but they are used in all of the wrong places, particularly in places where they don't belong because juniors and now AIs can recite them and want to use them everywhere. That's not even discussing whether the stable set itself is correct or not - it's dubious at this point.
Violets are blue
AI is great
And so are you
(None of above is theoretical)
Imagine the year is 1995, C exists, but some guy out there is working on essentially what modern Python is. He says to you "check out this language, you can just import stuff, and use it and dynamically modify anything at run time". You can probably come up with hundreds of arguments about things that could go wrong, like memory clean up, threading, e.t.c, but turns out, incrementally, they were all solved and we have the modern Python that basically is good enough to build these large LLM models.
Now imagine modern programming and computing is what C was back in 1995, and AI use is that guy building the Python code.
I think you have some serious misunderstanding here.
The infill will look seamless.
And entirely lack any actual strikes of interest - the outliers are exceptional signal and the entire raison d'etre for building such a database.
Jeez, if AI can just infill where the gold is, why even bother to look in the first place.
The original question was
>"clean up" dropped databases, compromised computers or leaked personal data?
For each of those things, you can right now build an agent that handles all of that. Or use a large frontier model with enough context to build code that ensures all of those edge cases are handled.
Future coding will essentially be like this. The concepts of dynamic vs compiled language will shift towards having frontier edge models put together code versus small runtime edge models dynamically processing data.
Also, Python does not build or run large language models. It orchestrates C code that does that, and it was probably good enough to do that in 1998.
The biggest change that happened was that hardware kept getting better and it became feasible to use garbage-collected languages everywhere including really inefficient implementations like CPython.
That being said, 30 years later Python is still slow as shit even compared to other dynamic languages and runs into all kinds of scaling issues when used for anything serious. And everywhere that performance matters, software continues to be written in typed, compiled languages including C (but also C++, Rust, Go, etc.). Even in ML, Python chiefly acts as a thin wrapper and glue language for high performance CUDA libraries (aka C and C++).
So your historical analogy is mostly anachronistic.
In the future, you won't be dealing with strings, json, or apis. You will be importing agents, and giving them brief instructions, either in plain English or in some intermediate language higher than Python that is more brief. Wanna deal with database reliability ? Import database agent and give it brief instructions on what you want to manage. Just like you mention, right now Python is the wrapper for low level libraries, because everyone who is doing work in ML doesn't want to waste time making sure their C Cuda kernels compile. In the same way, nobody is going to care if they get the API headers right, or if their strings are correctly parsed when you can just invoke a dedicated LLM (which will likely be highly specialized small model able to run on local hardware) to do all that.
You can scream and cry as much as you want how that is bad, how its slow, but nobody is going to care because shit is going to get built faster. Ever notice how despite the massive layoffs across tech, there isn't service degradation in any sector? Good luck trying to sell your Rust skills in the future lol.
It doesn't know what mess you want to clean up. A lot of times AI just starts making up new patterns on top of other patterns and having backwards compatibility between the two. How does it know which one you actually like?
I thought the same when I saw development outsourced to Indians that struggled to write a for loop.
I was wrong.
It turns out that customers will keep doubling down on mistakes until they’re out of funds, and then they’ll hire the cheapest consultants they can find to fix the mess with whatever spare change they can find under the couch cushions.
Source: being called in with a one week time budget to fix a mess built up over years and millions of dollars.
It's really nowhere near as complicated as making distributed systems reliable. It's really quite simple: read a fucking book.
Well, actually read a lot of books. And write a lot of software. And read a lot of software. And do your goddamn job, engineer. Be honest about what you know, what you know you don't know, and what you urgently need to find out next.
There is no magic. Hard work is hard. If you don't like it get the fuck out of this profession and find a different one to ruin.
We all need to get a hell of a lot more hostile and unwelcoming towards these lazy assholes.
Scrape off all the soil, put it in casks, and bury it in a concrete bunker for 10000 years. Then relocate everyone and attempt to rebuild.
We didn't create the dna we rely on to produce food and lumber, we just set up the conditions and hope the process produces something we want instead of deleting all the bannannas.
Farming is a fine an honorable and valuable function for society, but I have no interest in being a farmer. I build things, I don't plant seeds and pray to the gods and hope they grow into something I want.
If the farming situation were as dire as you seem to suggest, we'd have unpredictable famines all the time, but we don't
Planting is merely setting up the conditions. We didn't write the dna, we couldn't write the dna if we wanted to because we are an infinity away from understanding all the actual processes that descend from the dna. And when we utilize the dna that we simply found and didn't and couln't hope to write, it's always, at best, a case of hoping it goes right again this time.
Even when it works, even if you put in a lot of work and experience and understanding, it still just worked by itself and it's just good luck every time.
You have also guessed incorrectly.
In their current forms, it's unlikely for a product that actually needs to work.
It's not getting that complex and working with current LLMs.
The issues have all been structural, not local. It's easier to treat it like a rewrite using the original as a super detailed product spec. Working on the existing codebase works, but you have to aggressively modularize everything anyway to untangle it rather than attack it from the top down.
All of these projects have gone well, but I haven't run into a case where a feature they thought was implemented isn't possible. That will happen eventually.
It's honestly good, quick work as a contractor. But I do hope they invest in building expertise from that point rather than treating it like a stable base to continue vibecoding on.
The greatest asset in this type of work is genuinely liking people, being good at what you do, and keeping in touch. My email is easily findable for a reason.
The one part I do wonder is how to "keep in touch". Maybe it's a generational thing as a young millennial (some would call it "Zillenial") but the biggest issue in my networking over the year (cough and the dating scene cough) is ghosting. You think you hit it off, try to follow up the day after, and proceed to never again hear from them.
But won’t those more complex systems presumably solve more complex problems than the systems that humans could build? Or within a comparable time?
I think it is reasonably safe to assume at this point in the game that these AI systems are increasingly able to reason rigorously about novel problems presented to them, of ever increasing complexity and sophistication.
Ultimately, if you want to move fast, it's better just to have one engineer vibe coding something. but, that engineer is under so much pressure. Now he's got a legacy mode and another legacy mode because the requirements keep changing. And now there's a deadline in four weeks.
This all could work just fine, but the ungodly amount of attention that this world is getting puts too many cooks in the kitchen, which is always a recipe for disaster.
Purely AI written systems will scale to a point of complexity that no human can ever understand and the defect close rate will taper down and the token burn per defect rate scale up and eventually AI changes will cause on average more defects than they close and the whole system will be unstable. It will become a special kind of process to clean room out such a mess and rebuild it fresh (probably still with AI) after distilling out core design principles to avoid catastrophic breakdown.
Somewhere in the future, the new software engineering will be primarily about principles to avoid this in the first, place but it will take us 20 years to learn them, just like original software eng took a lot longer than expected to reach a stable set of design principles (and people still argue about them!).