Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
Because great insight and expertise stems from foundational knowledge.
I am a huge hockey fan. In the NHL, the players do not stop practicing the fundamentals once they make it to the NHL. They practice the fundamentals even more. Many practice the same drills as youth leagues -- stick handling, passing back and forth, shooting, edgework, shooting, rebound control, etc..
The best of the NHL might not hit the hardest, have the most accurate shot, or skate the fastest. What separates them from the rest of the lot is that they are fundamentally better than everyone else.
Transit routing is still moving fast in this area. We're still figuring out the best ways to return thousands-by-thousands transit time matrices with query-time truck dimensions and traffic updates. It can't be the only field!
With AI, people can build and collaborate on applications much more complex with much less technical knowledge. It might be ugly and crappy but I bet they’ll be mostly autonomous and not need to work through their IT team, or go through the hell of PM and requirements. If I know my requirements, I don’t need you. Hell, I can just start building and add requirements as I come across them. It’s not a major risk to the project like it used to be.
It will, and is, going much faster than the spreadsheet did.
Not to mention we told everyone to not roll their own anything, just use the free library for, clocks, time, crypto, auth, IPC, etc!
But that isn't the real problem here. Computer Science was always a theoretical concept, really. The meat of the issue is the rug being pulled out from the juniors and mids.
Reading code and writing it are two different skills. I believe that both are needed to maintain a codebase. It is much, much easier to understand what a service is doing when you're actually writing code that supports it, just like it's easier to remember the contents of a report that you wrote versus one that you read.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
I've had a lot success at the company I work for shipping small contained microservices for less technical teams to maintain with agents.
Personally I think this most of what SWEs should be doing these days... Code is easy to produce now. We should be handing over the creation and shipping of code to the relevant teams.
Maybe there will be more complexity overall, but it will be constrained via well designed microservices and distributed across the organisation.
Current project has over 60,000 LOC, I find using an antagonistic agent code and architecture reviews are really handy for keeping things grounded.
I do very little actual reading of code anymore. Now I am more of a results merchant.
I just did the LoC measure that isn't "official" but actually closer to how AI would read it: I.e. simply actual lines. Never mind "context window".
Just in the actual source folder there are 1.9 million lines. That's not tokens. Lines. Not even LoC as per some official canonicalized definition. Various languages. ~15 year old code base.
We work with Claude just fine.
For variable measures of "fine" depending on how much devs just trust whatever Claude spits out ;)
At the same size I only heard of AI generated projects reimplementing compilers and browser engines which come with amazing tests written by hand, but even these projects generally don't become reliable enough to use.
That 1.9m code base, within the same time frame as the 160k was created? <10k by the same measure.
10k I can comfortably throw at a local agent on my system (see recent posts in my history). Whether I can keep it that small is another question I suppose lol!
Is there ANY field where good engineers don't understand the basics? Can an electrical engineer do any work without understanding Volts and Amperes?
The LLM will look through the codebase, think, and tell you what it would do and if there's any design decisions you would have to make, as well as other things you probably need to be aware of. Then you go through the LLM's output and address all those decisions point by point, asking the LLM more questions if something isn't clear or requires more investigation or you're unsure what to do. And end with "address what I wrote, and share any other thoughts or questions or things to clarify you might still have, don't implement yet"
And you do this back and forth until all the design decisions have been addressed and you feel confident of what the code and architecture will look like, and only then say "ok, implement"
That way you get a lot of the benefits of writing it by hand (being forced to think through what the best design would be and how it would integrate with existing code, and increasing understanding of how existing code works) but it's still much faster. The tool I personally use is Cursor in auto mode.
P.S. actually before even that you first ask the LLM "what is the current state of X in the codebase" and then you ask follow-up questions until you have a good understanding of all the details that are relevant to you. And then you can start having the design/implementation conversation in the same chat context, since having the above information in context is useful.
https://www.aihero.dev/skills-grill-me
https://github.com/obra/superpowers
When I'm doing something quite complex I'll also throw in gstack office hours but I find that its usually overkill for many things
https://github.com/garrytan/gstack
I do often use the /qa skill regardless though. I always hated QA because it was so monotonous and it required a level of organization and creativity i never had (i mean im the developer so if I thought of edge cases id have solved for them)
I like that /qa covers and documents every edge case that it tested and I can nudge it about other ones
I find AI agents really great for codebase exploration and understanding how it works. In some ways even moreso than from manual implementation since it's easier to get a global picture.
You can also ask it questions like "are there recurring patterns of how X is done in the codebase?" to which it might answer sometime like "there are actually three competing patterns" and tell you what they are and the exact files, and then you can choose to refactor them if you want.
This is not non-coding work. This is coding.
My claim is just that LLMs are basically fine now with the "coding" part. Not so much with development, definitely not with engineering.
Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
Where they still fall massively short, even compared to a straight out of college junior developer, is figuring out what the correct question to ask is.
This has led to situations like Fable 5 spending six days debugging a deadlock because it didn't think to add any observability to detect when the job stalls, and would wait until the job timed out each time. Any junior developer would have fixed it in a few mins, but I left Fable running out of curiosity to see how long it'd take.
If anything I feel I am understating the capability of current models here. What is stopping the next generation of models from closing the feedback loop further?
An explicit query returns a specific answer, but means nothing about whether it can use that knowledge to create better software.
"Median" is kinda hard to judge given how rapidly the number of software engineers has been growing, how imprecise our within-industry skill assessments are, and also how fast everything within software was shifting even before agentic coding. Two examples: (1) an old estimate I heard was that 50% of the skills we would put on our CVs would be obsolete every few years, but we'd never know which ones; (2) The reason for the creation of FizzBuzz as an interview question (far too many people are *that* bad).
For the LLMs themselves: the METR time horizons estimates are based on an expert developer's wall-clock times, and the graph on their website only goes as far as Mythos preview which is kinda about this level. Mythos preview shows only 80% success at tasks that would take a human 3h6m on average (95% confidence interval 1h37m-6h39m), which is evidence that yes, there is plenty that the models can't do. I naturally break down larger tasks to smaller ones, the agentic coding models I've used also attempt to do this, but seem to be worse at it than I am.
> It strikes me that models already have superhuman software architecture capabilities, they are simply underelicited.
While they can do better when elicited correctly (and the same is true for humans), they are too "lazy" by themselves to attempt self-elicitation.
The second part, just like the first, is also true for humans; but LLMs are so much cheaper than humans that this is a disappointment for the LLM to stop early and say "[Thought for 35m 24s] Here's your MVP, I've not done subtasks 35 onwards", even though it's an advantage for the humans to say "[Invoice for a 2 week sprint with a team of 5 developers and their manager for a total cost of $23,000] Here's your MVP, we can discuss which of subtasks 35 onwards you want in the next sprint".
I'm not sure where I'd rate LLMs on a strict 0-10 scale from "hack" to "formal methods followed by end-to-end testing", but the examples given by the research companies themselves suggest that even they are only at the "this is starting to get interesting" level for both architecture and their own idea of what "I have now completed this to a good quality" looks like, not at the "actually good" level for complete big projects.
> Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
The observation is that money and compute spent are not strongly correlated with the general quality of the models. If the correlation was strong then Meta would be near the top, while the multitude of open-weight Chinese models that keep popping to the top of leaderboards every couple of months would be jokes. Grok would have been better from the start and not suddenly gotten good around when they bought up the much smaller and lower-budget Cursor.
https://llm-stats.com/models/muse-spark-1.1
https://llm-stats.com/models/gpt-5.6-sol
https://llm-stats.com/models/kimi-k3
> Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
My experience has been that predictions of "soon" fare about as poorly as predictions of "never" in the context of AI. Regarding the failures of "soon" predictions, I do not know the degree to which this is company leadership BSing how good their stuff is, vs. Goodhart's law where they're fooled by the difference between their metrics and reality.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
They wouldn't be able to verify if the code is correct, but based on my testing, if the work is simple enough, it probably wouldn't matter much.
There are a lot of companies whose modus operandi is basically "build the same basic site 300 times, except with a different colour scheme and logo".
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
I was just doing that yesterday designing a game demo lol
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
Human test passed. AIs make giant flashy mistakes. They don't fuck up basic grammar.
write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
Conveying the errors typical in a Raise The Colours group on Facebook, that's the hard stuff.
Sometimes they do. From Gemini 3, some AppleScript:
if something then
if something then
if something then
stuff
end if
end if
</if>
After about 10 turns the problem went away.
took complex pub/sub event architectures down to the events postgres can do on its own, and a system architecture that works with that payload. am aware of the limits with this approach and stress potential on db
Horrible if you got into this industry because you loved to code.
Amazing if you only ever saw the code as a means to an end.
Horrible if you got into engineering because you wanted to solve problems.
Amazing if you only ever wanted to see the problem solved.
Unfortunately, this won't be captured by the metrics, because its not easily observable or instrumentable. There will be relentless pressure to use AI and more and more and prove that it is adding value. It won't add value, but whats available to executives is to lower wages and employment. There will be 'success' stories by most VPs that they've been using AI and here is proof (low wages + employment) and trillions of lines of code. LOC has always been a useless metric, but nobody cares.
> At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
I don't think there will be more automation (at least not in a productive, helpful way), but a lot more enshittification and success stories of AI. Nobody cares about consumer experience or who is actually using the product.
Almost all CEOs/C-Suite/VPs are far removed from reality, they will do whatever is being hyped right now. Every Corp has had massive DEI initiatives in the previous administration and now anti-DEI in the current administration. Corps were doing blockchain nonsense and now they do AI.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.