To give an example from a field where LLMs started causing employment worries earlier than software development: translation. Some translators made their living doing the equivalent of routine, repetitive coding tasks: translating patents, manuals, text strings for localized software, etc. Some of that work was already threatened by pre-LLM machine translation, despite its poor quality; context-aware LLMs have pretty much taken over the rest. Translators who were specialized in that type of work and too old or inflexible to move into other areas were hurt badly.

The potential demand for translation between languages has always been immense, and until the past few years only a tiny portion of that demand was being met. Now that translation is practically free, much more of that demand is being met, though not always well. Few people using an app or browser extension to translate between languages have much sense of what makes a good translation or of how translation can go bad. Professional translators who are able to apply their higher-level knowledge and language skills to facilitate intercultural communication in various ways can still make good money. But it requires a mindset change that can be difficult.

1. No one cares about quality. Even in fields you'd expect to require the 'human touch' (e.g. novel translation), publishers are replacing translators with AI. It doesn't matter if you have higher-level knowledge or skills if the company gains more from cutting your contract than it loses in sales.

2. Translation jobs have been replaced with jobs proofreading machine translations, which pays peanuts (since AI is 'doing most of the work') but in fact takes almost as much effort as translating from scratch (since AI is often wrong in very subtle ways). The comparison to PR reviews makes itself.

After all, this has been Apple strategy since the 80's, and, even though there were some up's and down's, overall it's a success.

Maybe, but it probably requires a very strong and opinionated leader to pull off. The conventional wisdom in American business leadership seems to be to pursue the lowest level of quality you can get away with, and focus on cutting costs. And you'll have to fight that every second.

I don't think that's true at the individual-contributor level (pursing quality is very motivating), but they people who move up are the ones who sound "smart" by aping conventional wisdom.

> After all, this has been Apple strategy since the 80's, and, even though there were some up's and down's, overall it's a success.

I might give you that "since the late 90s," but there have been significant periods where that wasn't true (e.g. the early mid-90s Mac OS was buggy and had poor foundations).

AI needs documentation, automation, integration tests... It works very well for remote first company, but not for in-face informal grinding approach.

Just year ago, client told me to delete integration tests, because "they ran too long"!

Why are you surprised customers don't like spending money on the items that don't add business value. Add to that QA, documentation, security audits, etc.

They want to ship stuff that brings in customers and revenue day one, everything else is a cost.

Not to the non-technical bean counters. When they allocate money they want to see you prove how that extra money translates to an immediate ROI, and it's difficult to prove that in an Excel sheet exactly what the ROI will be without making stuff up on vibes and feels.

Like at one German company i was at ~15 years ago, all the devs wanted a second 19" monitor on our workstations for increased productivity, and the bean counters wouldn't approve that because they wanted proof of how that expense across hundreds of people will increase our productivity and by how much %, to see if that would offset the cost.

This is how these people think. If you don't bring hard numbers on how much their "line will go up", they won't give you money.

I know this is difficult to understand from the PoV of SV Americans where gazillions of dollars just fall from the sky at their tech companies.

A) how old the product is: Twitter during its first 5 years probaby had more work to do compared to Twitter after 15 years. I suspect that is why they were able to get rid of so many developers.

B) The industry: many b2c / ecommerce businesses are straightforward and don't have an endless need for new features. This is different than more deep tech companies

I’ve built several of such tools where I work. We don’t even have a dev team, it’s just IT Ops, and all of what I’ve built is effectively “done” software unless the business changes.

I suspect there’s a lot of that out there in the world.

On a macro level, if you were in a rising economic tide, you would still be hiring, and turning those productivity gains into more business.

I wonder what the parallels are to past automations. When part producing companies moved from manual mills to CNC mills, did they fire a bunch of people or did they make more parts?

- AI tools are expensive so until the increased productivity translates to increased revenue we need to make room in the budget

- We expect the bottlenecks in our org to move from writing code to something else (PM or design or something) so we're cutting SWEs in anticipation of needing to move that budget elsewhere.

- We anticipate the skillsets needed by developers in the AI world to be fundamentally different from what they are now that it's cheaper to just lay people off, run as lean as possible, and rehire people with the skills we want in a year or two than it is to try and retrain.

I don't necessarily agree with those arguments (especially the last one), but I think they're somewhat valid arguments

> rehire people with the skills we want in a year or two than it is to try and retrain.

before that future comes your company might become obsolete already, because you have lost your market share to new entrants

> We expect the bottlenecks in our org to move from writing code to something else

I would love to tell them, hey lets leverage current momentum and build, when those times come, we offer existing people with accumulated knowledge to retrain to a new type of work, if they think they're not good fit, they can leave, if they're willing, give them a chance, invest in people, make them feel safe and earn trust and loyalty from them

> AI tools are expensive so until the increased productivity translates to increased revenue we need to make room in the budget

1. Its not that expensive: 150$/seat/month -> 5 lunches? or maybe squeeze it from Sales personnel traveling with Business class?

2. By the time increased productivity is realized by others, company who resisted could be so far behind, that they won't be able to afford hiring engineers with those skillsets, if they think 150$ is expensive now, I am sure they will say "What??? 350k$ for this engineer?, no way, I will instead hire contractors"

However AI definitely is capable of lower end software tasks and really well trodden ground, especially when managed by a developer, so perhaps what we will see is a bigger gap in pay and talent not too different from the off-shore vs on-shore market comparisons.

The key for me though for me is that, if AI makes your employees 20% more valuable, that will either get priced into their wage or captured by the business, but it still doesn't replace the need for good talent (software engineer, agent handler, whatever it will get called).

Jobs where a machinist is in charge of large chunks of the process are rarer. Large shop will have one person setting up many machines to maximize throughput.

If it is truly because of AI, then it's still a losing strategy long term in my opinion.

It’s refreshing to see the same sentiment from so many other people independently here.

That's one of the reasons why I am terrified, because it can lead to burn out, and I personally don't like to babysit bunch of agents, because the output doesn't feel "mine", when its not "mine" I don't feel ownership.

And I am deliberately hitting the brake from time to time not to increase expectations, because I feel like driving someone else's car while not understanding fully how they tuned their car (even though I did those tunings by prompting)

The engineer who worked with you took ownership of the code! Have you forgotten this?

It's not going to be written exactly like you would do it, but that's ok - because you care about the results of the solution and not its precise implementation. At some point you have to make an engineering decision whether to write it yourself for critical bits or allow the agent/junior to get a good enough result.

You're reviewing the code and hand editing anyway, right? You understand the specs even if your agent/junior doesn't, so you can take credit even if you didn't physically write the code. It's the same thing.

Yes, yes!

And this is problem for me, because of the pace, my brain muscles are not developing enough compared to when I was doing those things myself.

before, I was changing my mind while implementing the code, because I see more things while typing, and digging deeper, but now, because juniors are doing things they don't offer me a refactoring or improvements while typing the code quickly, because they obey my command instead of having "aha" moment to suggest better ways

Doesn't exclude the possibility of short term distribution, though.

It might not be about bringing more revenues but retaining market share.

Once you get to a certain size company, this means a lot of bloat. Heck, I've seen small(ish) companies that had as many managers and administrators as ICs.

But You're not wrong, I'm just pointing out how an org that has 4k people can lay off a few hundred with modest impact of the financials (though extensive impact on morale).

If you look at my post history I'm essentially saying the same stuff lol.

If, and it's a big if, AI models really boost productivity by an order of magnitude (I personally, while being skeptical a year or two ago, am leaning towards this idea) then engineers have a chance to realize their ideas, improve current system design patterns and build successful companies, which will inevitably (hopefully) require hiring personnel to keep competing, bringing entire software engineering market to a newly balanced state.

On the other hand, our corporate AI is.. not great atm. It was briefly kinda decent and then suddenly it kinda degraded. Worst case is, no one is communicating with us so we don't know what was changed. It is possible companies are already trying to 'optimize'.

I know it is not exactly what you are asking. You are saying capability is there, but I am personally starting to see a crack in corporate willingness to spend.

The group used feature flags...

    if (a) {
       // new code
    } else {
       // old code
    }

    void testOff() {
       disableFlag(a);
       // test it still works
    }
    
    void testOn() {
        enableFlag(a);
        // test it still works
    }

I can do one flag in ~3minutes. Code edit, pr prepped and sent.

The AI can do one in 10mins, but I couldn't look away. It kept trying to use find/grep to search through a huge repo to find symbols (instead of the MCP service).

Then it ignored instructions and didn't clean up one or the other test, left unused fields or parameters and generally made a mess.

Finally, I needed to review and fix the results, taking another 3-5 minutes, with no guarantee that it compiled.

At that point, a task that takes me 3 minutes has taken me 15.

Sure, it made code changes, and felt "cool", but it cost the company 5x the cost of not using the AI (before considering the token cost).

Even worse, the CI/CD system couldn't keep up the my individual velocity of cleaning these up, using an automated tool? Yeah, not going to be pleasant.

However, I need to try again, everyone's saying there was a step change in December.

Claude Code took 4 hours, with multiple prompts. At the end, it started to break the previous fixes in favor of new features. The code was spaghetti. There was no way I could fix it myself or steer Claude Code into fixing it the right way. Either it was a dead-end or a dice roll with every prompt.

Then I implemented my own version with Cursor tab completion. It took the same amount of time, 4 hours. The code had a clear object-oriented architecture, with a structure for evolution. Adding a new feature didn't require any prompts at all.

As a result, Claude Code was worse in terms of productivity: the same amount of time, worse quality output, no possibility of (or at best very high cost of) code evolution.

Me too. After listening to all the claims about Claude Code's productivity benefits, I was surprised to get the result I got.

I'm not able to share details of my work. I was using Claude Opus 4.5, if I recall correctly.

I wanted it to finish up some tests that I had already prefilled, basically all the AI had to do was convert my comments into the final assertions. A few minutes later of looping, I see it finishes and all tests are green.

A third of the tests were still unfilled, I guess left as an exercise for the reader. Another third was modified beyond what I told it to do, including hardcoding some things which made the test quite literally useless and the last third was fine, but because of all the miscellaneous changes it made I had to double check those anyways. This is about the bare minimum where I would expect these things to do good work, a simple take comment -> spit out the `assert()` block.

I ended up wasting more time arguing with it than if I had just done the menial task of filling out the tests myself. It sure did generate a shit ton of code though, and ran in an impressive looking loop for 5-10 minutes! And sure, the majority of the test cases were either not implemented or hardcoded so that they wouldn't actually catch a breakage, but it was all green!!

That's ultimately where this hype is leading us. It's a genuinely useful tool in some circumstances, but we've collectively lost the plot because untold billions have poured into these systems and we now have clueless managers and executives seeing "tests green -> code good" and making decisions based on that.

We can now make 1$ million dollar commercials with 100,000$ or less. So a 90% reduction in costs - if we use AI.

The issue is they don’t look great. AI isn’t that great at some key details.

But the agencies are really trying to push for it.

They think this is the way back to the big flashy commercials of old. Budgets are lower than ever, and shrinking.

Big issue here is really the misunderstanding of cause - budgets are lower, because advertising has changed in general (TV is less and less important ) and a lot of studies showed that advertising is actually not all that effective.

So they are grabbing onto a lifeboat. But I’m worried there’s no land.

I’ve planned my exit.

Also what are you existing to?

When the metrics arrived with digital, they saw that advertising, in some ways, was just not as effective as they’d hoped. In some ways the ROI wasn’t there. Seth Godin agrees. He says that advertising in the digital era could be as simple as just having a good product. I think this is Tesla’s position on it - make the best product and the internet takes care of it.

Legacy companies have kept large ad budgets but those are diminishing. From what I spoke with my friend at WPP, he said their data science team showed that outside of a new product or a product that is not recognised by consumers, the actual outcomes from ads are marginal or incremental. Thats what he told me. If your product is already known to consumers, the ROI is questionable.

There’s something about AIs that feels wrong for storytelling. I just don’t think people will want AIs to tell them stories. And if they do… well not my concern. I believe in human storytelling.

At review time.

There are simply too many software industries that can't delegate both authorship _and_ review to non-humans because the maintenance/use of such software, especially in libraries and backwards-compat-concerning environments, cannot justify an "ends justifies the means" approach (yet).

I also heard an opinion that since writing code is cheap, people implement things that have no economic value without really thinking it through.

Only it doesn't, there's product positioning, UX, information architecture, onboarding and training, support, QA, change management, analytics, reporting… sigh

Tip to budding software engineers: try to not work in these sort of places, as they're about "looking busy" rather than engineering software, where the latter is where real long-lasting things are built, and the former is where startup founders spend most their money.

The last paragraph is where the tricky and valuable parts are, and also where AI isn't super helpful today, and where you as a human can actually help out a lot if you're just 10% better than the rest of the "engineers" who only want to ship as fast as possible.

Isn't it a very inefficient way to learn things? Like, normally, you would learn how things work and then write the code, refining your knowledge while you are writing. Now you don't learn anything in advance, and only do so reluctantly when things break? In the end there is a codebase that no one knows how it works.

In a legacy codebase this may require learning a lot of things about how things work just to make small changes, which may be much less efficient.

It is. But there are 2 things:

1. Do I want to learn that? (if I am coming back to this topic again in 5 months, knowledge accumulates, but there is a temptation to finish the thing quickly, because it is so boring to swim in huge legacy codebase)

2. How long it takes to grasp it and implement the solution? If I can complete it with AI in 2 days vs on my own in 2 weeks, I probably do not want to spend too much time on this thing

as I mentioned in other comments, this is exactly makes me worried about future of the work I will be doing, because there is no attachment to the product in my brain, no mental models being built, no muscles trained, it feels someone else's "work", because it explores the code, it writes the code. I just judge it when I get a task

It pains the anti-capitalist fibers in my body to say this, but no they are not. At the maximum the value is in organizational knowledge and existing assets (= source code, documentation), so that people with the least knowledge possible can make changes. In software companies in general, technical excellence and knowledge is not strongly correlated with economic success as long as you clear a certain bar (that's not that high). In comparison, in hardware/engineering companies, that's a lot more correlated.

In the concrete example of a legacy codebase we have here, there is even less value in trying to build up knowledge in the company, as it has already been decided that the system is to be discarded anyways.

LLMs also are quite bad for security. They can find simple bugs, but they don't find the really interesting ones that leverage "gap between mental model and implementation" or "combination of features and bugs" etc, which is where most of the interesting security work is imo.

I am doing novel work with codex but it does need some prompting ie. exploring possibilities from current codebase, adding papers to prompt etc.

For security, I think I generally start a new thread before committing to review from security pov.

If you are relying on the LLM and context, then unless your context is a secret your competitor is only ever one prompt behind you. If you're willing to pursue true novelty, you need a human and you can leap beyond your competition.

The reality is that a huge portion of my time is spent doing similar work and what LLMs largely do is pick up the smaller tasks or features that I may not have prioritized otherwise. Revolutionary in one sense, completely banal and a really minor part of my job in many others.

https://aisle.com/blog/what-ai-security-research-looks-like-...

  People who are saying they're not seeing productivity boost, can you please share where is it failing?

I'm going to guess that many people on HN are also on the "free ChatGPT isn't that good at programming" train.

Probably that's the reason why some people are sure their job is still safe.

Nature of job is changing rapidly

I'm not a tech CEO but people who are anti-LLM for programming have no place on my team.

I’m reminded of https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...

How much bloat and bureaucracy bottleneck is sitting in middle management whose favorite past time is wasting everyone's time on meetings that could have been an email? HR? Not the execs, but the HR drones that do nothing but answer employee questions about policy, could have already been replaced with not even an AI, just an old school chatbot, a long time ago.

Instead of cutting engineers, cut the non-tech jobs, flatten the structure.

There’s obviously a benefit of paying higher rates for US programmers, but does that benefit change when llms are thrown into the mix

It takes more planning, more specification, more coordination, more QA. The quality is almost always worse, and remediation takes forever. So your BA, QA and PM time goes way up and absorbs any cost savings.

YMMV.

Someone deciding to drop a spreadsheet of customer data into their personal AI account to increase their productivity would be catastrophic for business, so you need rules. And rules means paying for enterprise AI tooling.

The $20 a month tier in particular is a trivial expense, on par with businesses that expect their workers to wear steel toed shoes. Some may give workers a little stipend to buy those boots, some not. Either way, it doesn't really matter.

I don't do tech outside of 9-5, so either my employer pays for it all, or I don't use it. Simple as that. Thankfully, they do pay for it, but I couldn't imagine working somewhere that says "You need to use AI" and then not providing it on their dime.

Quite frankly it should be regulation that if a W2 employee needs something to perform their job duties, the employer must provide it.

FWIW I find it useful if I know exactly what I want and it's quicker to prompt it than type it myself. Also for research and building understanding it's generally good. I still catch it being wrong on details of you're really paying attention or literally contradicting itself between prompts. That gives me a lot of pause about trusting things it told me that I just accepted as fact without having enough knowledge myself to question it.

Til then wtf_are_these_abstractions.jpg

I worry that we're returning to an era of renting core development tools. After the huge benefits from free and open source tools, that's a bitter pill to swallow.

The question is really, velocity _of what_?

I got this from a HN comment. It really hit for me because the default mentality for engineers is to build. The more you build the better. That's not "wrong" but in a business setting it is very much necessary but not sufficient. And so whenever we think about productivity, impact, velocity, whatever measure of output, the real question is _of what_? More code? More product surface area? That was never really the problem. In fact it makes life worse majority of the time.

They've already admitted they just 'throw the code away and start again'.

I think we've got another victim of perceived productivity gains vs actual productivity drop.

People sitting around watching Claude churn out poor code at a slower rate than if they just wrote it themselves.

Don't get me wrong, great for getting you started or writing a little prototype.

But the code is bad, riddled with subtle bugs and if you're not rewriting it and shoving large amounts of AI code into your codebase, good luck in 6-12 months time.

Edit: Ha, and the report claims it's relatively good at business and finance...

Edit 2: After discussion in this thread, I went back to opus and asked it to link to articles about how to handle non-normally distributed data, and it actually did link to some useful articles, and an online calculator that I believe works for my data. So I'll eat some humble pie and say my initial take was at least partially wrong here. At the same time, it was important to know the correct question to ask, and honestly if it wasn't for this thread I'm not sure I would have gotten there.

A good way to use AI is to treat it like a brilliant junior. It knows a lot about how things work in general but very little about your specific domain. If your data has a particular shape (e.g lots of orders with a few large orders as outliers) you have to tell it that to improve the results you get back.

- https://cxl.com/blog/outliers/

- https://www.blastx.com/insights/the-best-revenue-significanc...

- (online tool to calculate significance) https://www.blastx.com/rpv-calculator

I'm not checking their math, but the articles make sense to me, and I trust they did implement it correctly. In the end the LLM did get me to the correct answer by suggesting the articles, so I guess I should eat some humble pie and say it _did_ help me. At the same time, if I didn't have the intuition that using rpv as-is in a t-test would be noisy, and the suggestions from this comment thread, I think I could have gone down the wrong path. So I'm not sure what my conclusion is -- maybe something like LLMs are helpful once you ask the right question.

(I also just love statistics and think it's some of the most applicable math to everyday life in everything from bus arrival times to road traffic to order values to financial markets.)

Tests were always important, but now they are the gatekeepers to velocity.

It’s fine at replacing what stack overflow did nearly a decade ago, but that isn’t really an improvement from my baseline.

This is what's so frustrating about the hype bros for me. In most cases, everything AI spits out are code smells.

We're all just supposed to toss out every engineering principle we've learned all so the owner class can hire less developers and suppress wages?

I'm sure it's working great for everyone working on SaaS CRUD or web apps, but it's still not anywhere close to solving problems outside that sphere. Native? It's very hit and miss. It has very little design sense (because, why would it? It's a language model) so it chokes on SwiftUI, it also can't stop using deprecated stuff.

And that's not even that specialized. It still hallucinates cmdlets if you try to do anything with PowerShell, and has near zero knowledge about the industry I work in, a historically not tech-forward industry where things are still shared in handcrafted PDF reports emailed out to subscribers.

I'm going to leave this field entirely if the answer just becomes "just make everything in React/React Native because it's what the AI does best."

I end up replacing any saved time with QA and code review and I really don’t see how that’s going to change.

In my mind I see Claude as a better search engine that understands code well enough to find answers and gain understanding faster. That’s about it.

The productivity gains are blatantly obvious at this point. Even in large distributed code bases. From jr to senior engineer.

Productivity is a term of art in economics and means you generate more units of output (for example per person, per input, per wages paid) but doesn't take quality or otherwise desireability into account. It's best suited for commodities and industrial outputs (and maybe slop?).

Yesterday a colleague didn’t quite manage to implement a loading container with a Vue directive instead of DOM hacks, it was easier for me to just throw AI at the problem and produced a working and tested solution and developer docs than to have a similarly long meeting and have them iterate for hours.

Then I got back to training a CNN to recognize crops from space (ploughing and mowing will need to be estimated alongside inference, since no markers in training data but can look at BSI changes for example), deployed a new version of an Ollama/OpenAI/Anthropic proxy that can work with AWS Bedrock and updated the docs site instructions, deployed a new app that will have a standup bot and on-demand AI code review (LiteLLM and Django) and am working on codegen to migrate some Oracle forms that have been stagnating otherwise.

It’s not funny how overworked I am and sure I still have to babysit parallel Claude Code sessions and sometimes test things manually and write out changes, but this is a completely different work compared to two or three years ago.

Maybe the problem spaces I’m dealing with are nothing novel, but I assume most devs are like that - and I’d be surprised at people’s productivity not increasing.

When people nag in meetings about needing to change something in a codebase, or not knowing how to implement something and its value add, I’ll often have something working shortly after the meeting is over (due to starting during it).

Instead of sending adding Vitest to the backlog graveyard, I had it integrated and running in one or two evenings with about 1200 tests (and fixed some bugs). Instead of talking about hypothetical Oxlint and Oxfmt performance improvements, I had both benchmarked against ESLint and Prettier within the hour.

Same for making server config changes with Ansible that I previously didn’t due to additional friction - it is mostly just gone (as long as I allow some free time planned in case things vet fucked up and I need to fix them).

Edit: oh and in my free time I built a Whisper + VLM + LLM pipeline based on OpenVINO so that I can feed it hours long stream VODs and get an EDL cut to desired length that I can then import in DaVinci Resolve and work on video editing after the first basic editing prepass is done (also PyScene detect and some audio alignment to prevent bad cuts). And then I integrated it with subscription Claude Code, not just LiteLLM and cloud providers with per-token costs for the actual cuts making part (scene description and audio transcriptions stay local since those don't need a complex LLM, but can use cloud for cuts).

Oh and I'm moving from my Contabo VPSes to running stuff inside of a Hetzner Server Auction server that now has Proxmox and VMs in that, except this time around I'm moving over to Ansible for managing it instead of manual scripts as well, and also I'm migrating over from Docker Swarm to regular Docker Compose + Tailscale networks (maybe Headscale later) and also using more upstream containers where needed instead of trying to build all of mine myself, since storage isn't a problem and consistency isn't that important. At the same time I also migrated from Drone CI to Woodpecker CI and from Nexus to Gitea Packages, since I'm already using Gitea and since Nexus is a maintenance burden.

If this becomes the new “normal” in regards to everyone’s productivity though, there will be an insane amount of burnout and devaluation of work.

We've started building harnesses to allow people who don't understand code to create PRs to implement their little nags. We rely on an engineer to review, merge, and steward the change but it means that non-eng folks do not rely on us as a gate. (We're a startup and can't really afford "teams" to do this hand-holding and triage for us.)

As you say we're all a bit overworked and burned out. I've been context switching so much that on days when I'm very productive I've started just getting headaches. I'm achieving a lot more than before but holding the various threads in my head and context switching is just a lot.

I've always done more in days than others might in a week. YMMV.

1: pre-AI. Not keen on becoming a manager of an idiot savant, so I’m planning my exit.

Note1: I have "expert" level research skills. But LLMs still help me in research, but the boost is probably 1.2x max. But

Note2: By research, I mean googling, github search, forum search, etc. And quickly testing using jsfiddle/codepen, etc.

But I also think you are overestimating your RESEARCH skills, even if you are very good at research, I am sure you can't read 25 files in parallel, summarize them (even if its missing some details) in 1 minute and then come up with somewhat working solution in the next 2 minutes.

I am pretty sure, humans can't comprehend reading 25 code files with each having at least 400 lines of non-boilerplate code in 2 minutes. LLM can do it and its very very good at summarizing.

I can even steer its summarizing skills by prompting where to focus on when its reading files (because now I can iterate 2-3 times for each RESEARCH task and improve my next attempt based on shortcomings in the previous attempt)

In this example, if the 25 files are organized nicely, and I had I nice IDE that listed class/namespace members of each file neatly, I might take 30 minutes to understand the overall structure.

Morever, If I critically analyzed this, I would ask "how many times does this event of summarizing 25 files happen"? I mean, are we changing codebases every day? No, it's a one time cost. Moreover, manually going through will provide insight not returned by LLM.

Obviously, every case is different, and perhaps you do need to RESEARCH new codebases often, I dunno!

* you probably lack good RESEARCH skills

* I can see at most 1.25x improvements - now it is 2-3x

By updating your comment you are making my reply irrelevant to your past response

If I can just vibe and shrug when someone asks why production is down globally then I'm sure the amount of features I can push out increases, but if I am still expected to understand and fix the systems I generate, I'm not convinced it's actually faster to vibe and then try to understand what's going on rather than thinking and writing.

In my experience the more I delegate to AI, the less I understand the results. The "slowness and thinking" might just be a feature not a bug, at times I feel that AI was simply the final straw that finally gave the nudge to lower standards.

You're pretty high up in the development, decision and value-addition chain, if YOU are the responsible go-to person for these questions. AI has no impact on your position.

Tangential, I don't even know what "responsible" in the corporate world means anymore, it seems to me no one is really responsible for anything. But the one thing that's almost certain is that I will fix the damn thing if I made it go boom.

God I hope I never ever have to work with you

I have never heard anyone say "it works" as a positive thing when reviewing code..

Yes, there is a productivity boost but you can't tell me there is no decrease in quality

Gaslight me by telling me I must be a time traveler because I use go 1.26 but the latest version actually is 1.24

And tell me I can't use wg.Go() because this function does not exist (it does)

However now that it's in the beta stage the amount of issues and bugs is insane. I reviewed a lot of the code that went in as well. I suspect the bug fixing stage is going to take longer than the initial implementation. There are so many issues and my mental model of the codebase has severely degraded.

It was an interesting experiment but I don't think I would do it again this way.

I came to the same conclusion when producing a video with Grok. Did the job but utterly painful and it was definitely very costly - I used 50 free-trial accounts and maxed them out each day for a month.

Im pretty sure these conclusions hold across all models and therefore the technology by extension.

Not only that, the less coding you do in general? Guess what, fixing issues that in the past wouldve been a doddle (muscle memory) become less harder due to atrophy.

Swear most people dont think straight and cant see the obvious.

Why? This is great. AI fixing up huge legacy codebases is just taking the jobs humans would never want to do.

I find anything else, I spend more time coaxing them into doing 85% of what I need that I'm better off doing it myself.

So they're not useless but there's only so many times in a week that I need a function to pretty-print a table in some fashion. And the code they write on anything more complex than a snippet is usually written poorly enough that it's a write-once-never-touch-again situation. If the code needs to be solid, maintainable, testable, correct (and these are kind of minimal requirements in my book) then LLMs make little impact on my productivity.

They're still an improvement on Google and Stack exchange, but again - only gets you so far.

YMMV

What was the last thing you built in which you felt this was the case?

You must be working in a very niche field with very niche functionality if that's the case? I work at a company just outside of FAANG and I work in compliance. Not a terribly complex domain but very complicated scale and data integrity requirements.

I haven't written a single line of code manually in 2 weeks. Opus 4.6 just... works. Even if I don't give it all the context it just seems to figure things out. Occasionally it'll make an architectural error because it doesn't quite understand how the microservices interact. But these are non-trivial errors (i.e. humans could have made them as well) and when we identify such an error, we update the team-shared CLAUDE.md to make sure future agents don't repeat the error.

What I got sort of works, as in tests pass - this with Opus 4.5. It is usable, though it doesn't exist cleanly on errors despite working to death with Claude about this. On exception it exits dirtily and crashes, which is good enough for now. I had some fancy ideas about logging messages from the vendor to be able to replay them, to be able to then reproduce errors. Opus made a real hash of it, lots of "fuck it comment out the assert so the test passes". This part is unusable and worse, pollutes the working part of the project. It made a valiant effort at mocking the vendor API for testing but really badly, instead of writing 30 lines of general code, it wrote 200 lines of inconsistent special cases that don't even work altogether. Asked to fix it it just shuffles around the special cases and gets stuck.

It's written messily enough that I wouldn't touch this even to remove the dead code paths. I could block a few days for it to fix but frankly in that time I can redo it all and better. So while it works I'm not gonna touch it.

I did everything LLM proponents say. I discussed requirements. Agent had access to the API docs and vendor samples. I said think hard many times. Based on this we wrote a detailed spec, then detailed inplementation plan. I hand checked a lot of the high level definitions. And yet here I am. By the time Opus went away and started coding, we had the user facing API hammered out, key implementation details (callback -> queue -> async task in source thread routing messages etc), constraints (clean handling of exceptions, threadsafe etc). Tests it has to write. Any minor detail we didn't discuss to death was coded up like a bored junior.

And this also wasn't my first attempt, this was attempt #3. First attempt was like, here's the docs and samples, make me a Python async API. That was a disaster. Second was more like, let's discuss, make a spec, then off you go. No good. Even just taking the last attempt time, I would have spent less time doing this by hand myself from scratch.

> lots of careful code of registering and unregistering callbacks, some careful thread synchronisation (callbacks get called in another thread), thinking about sane exception handling in async code. Fiddly but not rocket science.

I'd expect CC to fail this when just given requirements. The way I use it is to explicitly tell it things like: "Make sure to do Y when callback X gets fired" and not "you have to be careful about thread synchronisation". "Do X, so that Exceptions are always thrown when Y happens" instead of "Make sure to implement sane Exception handling". I think you have to get a feeling for how explicit you have to get because it definitely can figure out some complexity by itself.

But honestly it's also requires a different way of thinking and working. It reminds me of my dad reminiscing that the skill of dictating isn't used at all anymore nowadays. Since computers, typing, or more specifically correcting what has been typed has become cheap. And the skill of being able to formulate a sentence "on the first try" is less valuable. I see some (inverse) parallel to working with AI vs writing the code yourself. When coding yourself you don't have to explicitly formulate everything you are doing. Even if you are writing code with great documentation, there's no way that it could contain all of the tacit knowledge you as the author have. At least that's how I feel working with it. I just got really started with Claude Code 2 months ago and for a greenfield project I am amazed how much I could get done. For existing, sometimes messy side projects it works a lot worse. But that's also because it's more difficult to describe explicitly what you want.

At this point I'm basically programming in English, no? Trying to squeeze exact instructions into an inherently ambiguous representation. I might as well write code at this point, if this is the level of detail required. For this to work, I need to be able to say "make this thread-safe", maybe "by using a queue". Not explaining which synchronisation primitive to use in every last piece of the code.

This is my point actually. If I describe the task to accuracy level X, it still doesn't seem to work. To make it work, perhaps I need to describe it to level Y>X, but that for now takes me more time than to do it myself.

There's lots of variables here, how fast I am at writing code or planning structure, how close to spec the things needs to be, etc. My first "vibe code" was a personal productivity app in Claude Code, in Flutter (task timing). I have 0 idea about Dash or Flutter or any web stuff, and yet it made a complete app that did some stuff, worked on my phone, with a nice GUI, all from just a spec. From scratch, it would take me weeks.

...though in the end, even after 3 attempts, the final thing still didn't actually work well enough to be useful. The timer would sometimes get stuck or crash back down to 0, and froze when the app was minimised.

Yea, except they can handle some degree of complexity. Its usefulness obviously really depends on that degree. And I'm sure there are still a lot of domains and types of software where that tradeoff between doing it yourself or spelling it out isn't worth it.

I can see that there's a lot of applications where things can just randomly fail and you retry / restart, that helps with crashes.

But the AI can't make it not crash, what's to say it does the right thing when it succeeds? Again, depends on the relative cost of errors etc.

Silent failures and unexplained crashes are high on my list of things to avoid, but many teams just take them for granted in spite of the practical impact.

I think that a lot of orgs have a culture of "ship it and move on," accompanied by expectations like: QA will catch it, high turnover/lower-skill programmers commit stuff like this all the time anyway, or production code is expected to have some rough edges. I've been on teams like that, mostly in bigger orgs with high turnover and/or low engineering standards.

i think ive done some python on each major model release, not necessarily woth claude code, but it feels like if it touches python, it gets stupid, quickly.

and not just for the python, it stays stupid when it wants to say, write html. for a tiny data labeling tool, it turned off user select, for instance

1. Performance-critical code to featurize byte slices for use in a ML model. Claude kept trying to take multiple passes over the slice when the featurization can obviously be done in one. After I finally got it to do the featurization in one pass it was double-counting some bytes but not others (double counting all of them would have been fine since the feature vector gets normalized). Overall it was just very frustrating because this should have been straight-forward and instead it was dogshit.

2. Performance-critical code that iterates over lines of text and possibly applies transformations, similar to sed. Claude kept trying to allocate new Strings inside of the hot-loop for lines that were not transformed. When I told it to use Cow<'a, str> instead so that the untransformed lines, which make up the majority of processed lines, would not need a new allocation, Claude completely fucked up the named lifetimes. Importantly, my CLAUDE.md already tells Claude to use copy-on-write types to reduce allocations whenever possible. The agent just ignored it, which is _the_ issue with LLMs: they're non-deterministic and any guidance you provide is ultimately just a suggestion.

I can give you many, many examples of where it failed for me:

1. Efficient implementation of Union-Find: complete garbage result 2. Spark pipelines: mostly garbage 3. Fuzzer for testing something: half success, non-replicateable ("creative") part was garbage. 4. Confidential Computing (niche): complete garbage if starting from scratch, good at extracting existing abstractions and replicating existing code.

Where it succeeds: 1. SQL queries 2. Following more precise descriptions of what to do 3. Replicating existing code patterns

The pattern is very clear. Novel things, things that require deeper domain knowledge, coming up with the to-be-replicated patterns themselves, problems with little data don't work. Everything else works.

I believe the reason why there is a big split in the reception is because senior engineers work on problems that don't have existing solutions - LLMs are terrible at those. What they are missing is that the software and the methodology must be modified in order to make the LLM work. There are methodical ways to do this, but this shift in the industry is still in baby shoes, and we don't yet have a shared understanding of what this methodology is.

Personally I have very strong opinions on how this should be done. But I'm urging everyone to start thinking about it, perhaps even going as far as quitting if this isn't something people can pursue at their current job. The carnage is coming:/

I work at tech company just outside of big tech and I feel fairly confident that we won't have a need for the amount of developers we currently have within 3-4 years.

The bottleneck right now is reviewing and I think it's just a matter of time before our leadership removes the requirement for human code reviews (I am already seeing signs of this ("Maybe for code behind feature flags we don't need code reviews?").

Whenever there's an incident, there is a pagerduty trigger to an agent looking at the metrics, logs, software component graphs, and gives you an hypothesis on what the incident is due to. When I push a branch with test failures, I get one-click buttons in my PR to append commits fixing those tests failures (i.e. an agent analyses the code, the jira ticket, the tests, etc. and suggests a fix for the tests failing). We have a Slack agent we can ping in trivial feature requests (or bugs) in our support channels.

The agents are being integrated at every step. And it's not like the agents will stop improving. The difference between GPT3.5 and Opus 4.6 is so massive. So what will the models look like in 5 years from now?

We're cooked and the easiest way to tell someone works at a company who hasn't come very far in their AI journey is that they're not worried.

Sometimes I realise that this particular task has been slower than if I’d done it myself when I take in to account full wall clock time.

I can’t tell what type of task is going to work ahead of time yet.

I don't think features per hour is really what is holding back most established businesses.

My experiences suggest that we still have some time before the people that understand the plumbing of the business _and_ AI bubble up to positions of authority through wielding it practically and successfully at increasingly greater scale.