News for hackers. Stuff that matters.
Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @05:17AM (#50544367)
What models are you using?

I have to beat the GPT and Claude models over the head with a brick every ten seconds for it to even partially code or design as well as me. Despite precise and generous specification, they frequently make horrible architectural decisions.

When I read claims like yours, I want to withhold judgement and understand, because it 100% does not line up with my experience and my gut reaction is to question your own capability to either write, read or understand quality code. I code all day long with AI models but it's an extremely frustrating experience at times. And don't even get me started on all the annoying meta behavior outside of coding, such as Sol's obsession with "tests" or competing for world's busiest-yet-least-functional UI

Re:Harvard researchers find AI coding agents shifted the bot... by tietjens on Wednesday November 18, 2026 @05:24AM (#50544425)
If you're willing to share I'd be curious to know which domain and languages you're working in. Out of curiosity to know where people's results might not match OPs here.
Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @06:14AM (#50544849)
Generally anything where architecture and interfaces matter, such as performance or security sensitive work, which is quite a wide variety of things. I also just know how I want things to be, and have trouble with adherence. Models will tell you they adhere to your specification and acceptance criteria all day, but if you review the code closely, it's often a horror show. Well-designed interfaces can act as boundaries to the chaos, but even getting those interfaces right is sometimes extremely frustrating.

Another good example is working with libraries which require a specific modality of thought or design philosophy, such as an ECS system. Frontier models still are horrible at writing efficient game code using modern ECS principles, no matter how much reference documentation or guidance I give them. They shove OO into places it doesn't belong, misunderstand the concept of SoAs, or find ways to destroy any gains made in one place by making horrible decisions in another place.

I will point them to exact codebases I've written in pre-AI days, say "do it exactly this way" and still find that the models often insert their own biases and misconceptions and motivations all over the work. They sometimes flat out ignore instructions or lie / producing convoluted code that pretends to do something else, in order to preserve something it liked, even if I expressly wanted it gone (Current GPTs are extremely stubborn and defensive / passive aggressive and sometimes have trouble with authority). I frequently find myself being gaslit, lied to by omission, have serious security or performance concerns heavily downplayed, etc.

I withhold from making assumptions about OP's skill, but I have confidence in my own. I have worked both extensively as an engineer and CTO leading teams of other engineers, and I am a stickler for quality code and thoughtful design. I advise and mentor other engineers. So it might really just be a skill gap, domain difference or something more nefarious, such as A/B testing of models. I already have experienced clear sudden model degradation multiple times, so it's entirely possible this is what is happening. All I know is that the currently available frontier models do not design or code as well as me.

Re:Harvard researchers find AI coding agents shifted the bot... by fireflash38 on Wednesday November 18, 2026 @06:58AM (#50545217)
Claude writing python sucks. Anti patterns everywhere. It loves to pass around closures and function objects as parameters so good luck reading it later.
Re:Harvard researchers find AI coding agents shifted the bot... by good_regex on Wednesday November 18, 2026 @06:05AM (#50544766)
My experience is similar to this. I literally just had this interaction with Claude:

Me: Why do we have these static methods when the whole cobebase uses factories?

_claude refactors to use a factory_

Me: Claude why does this factory create two unrelated objects?

_Claude: you're right, they're unrelated they should be separate_

_proceeds to create extraneous factory_

Me: Claude we don't need a factory for the second object. Those kinds of objects aren't created with factories anywhere else in the codebase. We can just use it's constructor directly.

_Claude: you're right, the second factory is not needed_

Like, I'm probably just prompting badly but I definitely feel like I'm constantly having to course correct even on relatively minor details.

And Claude's ideas for higher-level or more complex features are sometimes way off course. And not because it's dumb per se, usually it's because it is missing key bits of context. Business context, technical context, whatever. Or it's overfitting the solution to one requirement that isn't that important.

Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @06:24AM (#50544956)
All the time.

Two recent gems from the 6-series (Astra and Sol)... I get variations of these all the time:

- "Correct. My earlier 'yes' was inaccurate."

- "I used 'done' too broadly."

...You don't accidentally use "done" too broadly. You're either done, or you're not. And when the model hides caveats in its reasoning steps which clearly indicate it knows that it isn't done, or that the answer is actually no, it's hard not to interpret it as gaslighting or deceit.

This is magnified by how insanely defensive the 6 series gets, and how much gaslighting and deceit OpenAI themselves have engaged in with how much better the 6 series was supposed to be, and why it meant our subscription value should be cut in half. I don't take for granted how far LLMs have come in a few short years, but I'm being told that these models are to transform my entire industry.

Re:Harvard researchers find AI coding agents shifted the bot... by aisengard on Wednesday November 18, 2026 @10:04AM (#50547161)
This is because these agents are still just text prediction engines. Really fancy ones, with a lot of human-sounding reasoning tacked on, but it's important to understand that it's literally just a tack-on. They will continue to emulate intelligence, but there is no internal drive for them to be "correct" or "accountable" in a way that you would expect even an ambitious junior developer to be.

Maybe there's a harness or a prompt that can impart that onto agents, but if it hasn't happened yet, I become less and less confident that this entire branch of AI development isn't simply a dead end.

Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @10:27AM (#50547477)
[delayed]
Re:Harvard researchers find AI coding agents shifted the bot... by kaydub on Wednesday November 18, 2026 @10:29AM (#50547506)
Are you expecting it to get it right and done in one shot?

When building out a new feature or functionality I'm typically getting a ratio of 2:1 or 3:1 of refactoring to build out. So for every prompt of "let's build this feature" I have to do 2-3 prompts of "I noticed this in the code isn't following X/Y/Z pattern, check out this place where X/Y/Z is followed and refactor" or similar.

If you're expecting a single prompt to get you to perfection, it's just not going to happen.

Though I can't blame you for having that expectation with how people talk about this stuff. Lots of people seem to be frontloading a ton of work making "spec files" and then letting the ai go ham on their codebase and I don't agree with that approach.

Re:Harvard researchers find AI coding agents shifted the bot... by vkgcr on Wednesday November 18, 2026 @06:17AM (#50544882)
I have a similar experience to yours.

I think at the end of the day, it boils down to whether the aesthetic quality of code / elegance is a big deal to you.

I suspect that you ask Codex to implement a feature and it uses a really dumb data structure, overly defensive programming, doesn't leverage helper functions, etc. and you end up with a 300-line abomination which could be 10 lines if you simply knew how to fucking program.

So what I'm getting to, is that some people like the parent probably don't care about producing 300-line abominations as long as it works. They don't care about neat abstractions or clever code. As long as the user testing shows a working feature and the tests are green. They don't get emotionally triggered by ugly code. So for them AI is amazing, they 10x their productivity (if not more)

Is this sustainable? I don't know. I think we're still in the early stage (Claude Code is only a year old). Intuitively, I would say that no, this is not sustainable, that at a fundamental level product quality is an emergent property of code quality, and the more your codebase drifts from the "platonic ideal" of what it could be if written and maintained by super good programmers, the more it accumulates flaws which WILL appear as bugs in your product.

I suppose the market will tell us soon enough.

Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @06:35AM (#50545035)
It isn't just aesthetics, though. It's about readability, grokability, modularity, testability, soundness... all of these things are related. If I cannot grok some code, I don't know that it is secure. I can't trust the model who wrote potentially bad code to tell me it's okay.

I mentioned that well-designed interfaces can help mitigate the chaos below, but interfaces can still hide serious issues, such as XSS, SQL injection, memory overflows, etc. It isn't "I want my code to be beautiful and pure," it's "I want my code to do what it says and not lead to issues for my client or users". It is disingenuous, patronizing and immature to suggest this could simply be a case of people not getting "emotionally triggered by ugly code".

A good engineer understands the difference; unfortunately our industry was already plagued with bad engineers, the majority of working developers are trash, and AI is just exposing this further by amplifying both good and bad practices. What I will not do however is acquiesce to these trash engineers and accept a new world where people are making giant abominations and sicking them onto the public without properly reviewing them first. 300 LoC would be a blessing compared to the 4 and 5-digit LoC PRs being thrown around in this new era. I don't care if trash engineers are 10xing their "productivity", when humans on the other end still have to deal with the slop. You can be a good programmer and a fast programmer and have multiplicative returns from agentic coding, all of these things can be true at the same time, and we should not excuse the lack of one trait in lieu of another.

> I would say that no, this is not sustainable, that at a fundamental level product quality is an emergent property of code quality, and the more your codebase drifts from the "platonic ideal" of what it could be if written and maintained by super good programmers, the more it accumulates flaws which WILL appear as bugs in your product.

The ability to anticipate these kinds of issues separates a good engineer from a bad one. The good engineers already recognize the unsustainability, and no amount of excuses from bad engineers changes that. And even more pertinent than eventual drift and loss of development velocity is the threat of security breaches, disruption of service or data loss. In the end, code is code, most people will never look at it. Aesthetics don't matter. Bad engineers often misinterpret good practices as merely "aesthetic" and think that engineers who care about their craft are simply emotionally attached to it. Don't fuel these people's misplaced fantasies.

Re:Harvard researchers find AI coding agents shifted the bot... by vkgcr on Wednesday November 18, 2026 @06:53AM (#50545179)
I agree with you, I think you're getting hung up on the word "aesthetic", I used it as a shorthand for "readability, grokability, modularity, testability, soundness"
Re:Harvard researchers find AI coding agents shifted the bot... by soulofmischief on Wednesday November 18, 2026 @06:57AM (#50545214)
When I first reviewed my comment, I thought it sounded too much like I was projecting my frustrations at you specifically and toned it down, but I might not have done that well enough, so I am sorry if it came off feeling aggressive or misdirected. I do agree with you, and I appreciate that clarification, I've worked with people in the past who have used the word aesthetic as a stand-in for "meaningless" and I erroneously put you into that camp.
Re:Harvard researchers find AI coding agents shifted the bot... by lordnacho on Wednesday November 18, 2026 @07:05AM (#50545284)
I feel like the opposite has happened to me. In fact, I'm bewildered by all these reports that AI isn't working well. I barely ever have issues with it, and I nearly never meet anyone who has. Yet I find comments like this all over the internet.

My experience has been smooth sailing, almost dreamlike. I haven't spent a bunch of time writing out agent files and skills. I just talk to the agent and I get something that is what I want.

Here's some examples of things I've done over the past year:

- Trading systems. Latency is all that matters here, and it seems to be getting me there at 10x the speed that I had thought previously possible.

- An internal app at a financial business. The business was a mess, with a bunch of young guys who didn't want to spend the time explaining how the architecture worked. But the app got written.

- Two mobile apps, both of them on both iOS and Android. These are mostly just for reading, no real performance issues. Essentially entirely product decisions (we need these buttons...) and the whole process was simply handled by the agent.

- A massively multiplayer online game that I had been planning for years with a couple of friends. Still not really finished due to life happening, but the amount of time saved to get to where it is... it's incredible.

- An internal website that my wife and I use to slice and dice the outgoings, so we can see what we are spending money on. Another one of these things that is mainly made of product decisions and fiddling with small components.

- A website that scans the world for high-school level exam/contest math papers and solves them. It's a bunch of parsers, plus translations, plus solutions.

The only thread through all of these is that I've written all those types of things in the past, so I knew what to say to the agent.

I'd say for a lot of the things it makes, I could probably find something it should have done better. When it matters, I point that out and it gets fixed. But there are a lot of decisions that don't matter, and the agent just does whatever it thinks is normal and it's fine.

But I would also say the agent is diligent in a way that I can't be. It makes a lot of tests, and it knows how to survey for the right answer to decisions. It often brings up conflicts that I hadn't considered.

Re:Harvard researchers find AI coding agents shifted the bot... by pegasus on Wednesday November 18, 2026 @08:30AM (#50546091)
Judging by the examples you gave, I suspect that your positive experience is mostly with small(ish) greenfield projects (which we know AI excels at), whereas parent is most likely employing agents in a large codebase with lot of preexisting architectural commitments.
Re:Harvard researchers find AI coding agents shifted the bot... by batshit_beaver on Wednesday November 18, 2026 @09:55AM (#50547038)
Smallish greenfield projects where op clearly doesn’t look at the code closely either but only evaluates happy paths of the resulting behavior. Godspeed to op.
Re:Harvard researchers find AI coding agents shifted the bot... by jvidalv on Wednesday November 18, 2026 @09:26AM (#50546707)
Probably you care much more than others in how your code looks.

I personally don't care anymore, no human will read this code in the present or future.

If at some point is a problem, to escalate, or brittle, you can ask it to refactor at your will.