Everyone is insane.

Just like many managers, the appearance of productivity is all that counts. And LLMs shine at giving the appearance of having solved all of the managers problems, and all they have to do to use it is spend on tokens.

This isn’t to say that LLMs aren’t truly useful, they absolutely are. But they’re very nature is one of simulating intelligence through next word prediction.

The chat modes and models are by their nature supremely attractive to management layers, because they give answers that sound so damn plausible even when they are complete fictions, and uttered with such confidence how could they be anything but the singularity.

Ymmv, do whatever you want . It's your life.

I shot down similar arguments in favor of outsourcing overseas for years. Outsourcing any critical logic to an LLM is even worse.

Outsourcing to another continent of humans and supplementing workflows with LLMs are entirely different operational universes. I think it is fair to put them on the same spectrum, but they're really far apart.

I'd argue outsourcing is a far more aggressive abdication of ownership of the technology than bringing an LLM agent in house and having it light a few fires under a few asses.

[edit] as the sibling points out, decayed system knowledge leads to relying on the LLM to fix the bugs the LLM introduced, which causes further decay in the institutional ability to reason with the business logic in code.

I've, for better or for worse, jumped full-on in with LLMs, and I can already feel my abilities radically declining over time. That's fine by me, but if you're a company looking to protect your institutional advantage - just handing over everything to an LLM suddenly means anybody can do exactly what you're doing, literally. At best and this is an absolute best case scenario, you're transferring your advantage to the stewardship of another company.

So the "knowledge base" an LLM generates is not useful as code, nor is it useful to humans. It may be useful to other LLMs as a lossy compression scheme for the original intention of the business logic and code.

That's not even in the same universe as institutional knowledge. Handing any serious business to something like that is malpractice, and sloppy beyond belief.

I'm not surprised that a lot of people think this way, because they never really grasped the benefit of holistic business knowledge united with code to begin with. Those people always outsourced, always got shitty code, and never really unified their systems. Cheap people and cheap companies take cheap exits. That's fine. But yes, many fragmented humans who all understand their portion in depth is much better than a bunch of markup files at retaining the knowledge of why things are done, procedurally, the way they are.

Once every six months the CEO calls me and asks me to remind him why our software does something like, idk, create a reverse payment instead of voiding a charge in some situation. Or some other thing he has asked a dozen times before. And I know the answer to why, or I know where to look, because I was in those meetings 5 years ago or because I wrote the code myself and asked the question when I wrote it.

Institutional knowledge is a very large context window, if you need to think of it that way, and LLMs are a shitty compression method for that. They can tell you how, but they don't reason well with why.

I think I know the reason to this one. Maybe. Because the money is already gone! There are no rollbacks in the banking system only "counter"-transactions.

The "if AI code breaks, who will fix it? -> "AI will fix it" exchange has (anecdotally) been very common among executives, which is much closer to outsourcing than programming.

Something completely different (but with the same logic): do you outsource legal, hire your own team of business lawyers or will you let customer services use AI for legal problems (and only hire a lawyer for a day in court)? I think all three solutions are currently active in different firms. From a risk perspective I would always want a lawyer on my team. Insource those learnings. But perspectives vary.

Moreover, any of us can run essential questions by each other, the marketing director, the lawyer, the CEO, the GM. And everyone has a stake. So all the creative and business decisions are worked out pretty quickly, and new features roll out with everything vetted and in place.

https://github.com/minimaxir/hacker-news-undocumented/blob/m...

Like: What competence do decision makers in FINANCE have, when they are this oblivious to economics?

I don't think this is axiomatic. I usually invest my time, energy and money in things that go squarely against the market's momentary sentiment.

It’s about pumping up revenues, earnings and eventually cash flows (since they can’t keep raising money constantly) to support a nonsensical valuation.

Tokens billed at API prices are profitable for openAI and anthropic today and it only get more lucrative every month for them as their inference costs fall. If it weren’t for continuous massive training runs taking larger and larger capex, these companies would be massively profitable

That's nice for them. What's less nice for them is that big customers are figuring out (some faster, some more slowly) that they don't want to pay API prices, or at least only for a much smaller number of tokens.

GPT-10 Human Ultra

But of course to some asshole non-technical people it meant asking for their vibe coded bullshit to be merged into production without review and fighting about it.

I am using gpt to populate JSON objects conforming to a list of natural language constraints for purposes of generating fake customers. I am finding that gpt5+ never fucks up. Not even a little bit. I've ran this test hundreds of times with 20+ constraints and it's been perfect every time.

Stable information yields stable control flow. Humans are much more likely to forget one of the many constraints during testing. This happy mistake may incidentally cover an edge but it also means we lose coverage elsewhere.

I think whether or not the LLM should be allowed to directly author deterministic control flow (code) is mostly the same thing. If you have a lot of constraints you want to satisfy all at the same time, this can give you a hit very close to the ideal target very quickly. Not knowing exactly what you want is when the LLM takes you for a ride.

1) Writing code the "old fashioned" way (i.e., a Python program that does X, Y, Z) allows you to arrive at a battle tested solution that will not change over time. From a risk assessment perspective, the behavior is essentially immutable, allowing a business to guarantee consistent behavior over long periods of time.

2) Just because something hasn't happened to you do, does not mean that it will not happen. LLM are opaque. If you stay on the "happy path", you may see consistent behavior for long periods of time, but there's always potential for an edge case where something goes catastrophically wrong. This is without even opening the can of worms regarding prompt injection and intentional sabotage of a working system.

3) There are plenty of real world examples of an LLM spontaneously deleting data from a DB (or the entire DB) or otherwise going completely off the rails. These might seem hyperbolic, but it happened at our company (to a test DB, not production). The severity of errors that occur can be existential to a business' survival without the proper guard rails.

4) There's no concrete way to truly confirm understanding between an LLM and a human. It can tell you that it completely understands what you want, and then it can do exactly the opposite. Followed by, "my bad" (Claude's new favorite catch phrase). Code can be audited and even proven to be correct given the appropriate level of time and energy.

My best results have been gleaned in using LLM to produce deterministic systems. I recognize everyone has different use cases and needs, but this seems to be the best use of the technology in my experience.

LLMs are probabilistic by design so running the same prompt multiple times will give you different results.

Otherwise, we wouldn’t needed LLMs and could replace it all with Postgres

Reasonably deterministic is the phrase. If I can be sure the LLM is giving me back the same result 99% of the time I need it, that's reasonable for me. Maybe this is not reasonable for others.

ie How often will an LLM get 2+2 wrong? Now expand until you're uncomfortable.

These models are great tools, but putting them and people on the same level does a disservice to our species and also is simply incorrect to what we know these models to be and their capabilities/limitations.

[0] https://www.theguardian.com/technology/2026/jun/01/meta-ai-h...

At the same time, one should acknowledge that not all tasks are on the same level.

That is to say I do agree that building reliable processes out of unreliable parts with feedback is the modus operandi. However AI cannot meaningfully handle feedback and learn. And that is a key unsolved problem.

If the Model makes repeated mistakes on the same subject matter, you can update your agent.md file, or you can add skills to deal with specific prompts, or you provide a better default harness.

The whole idea of coding agents is their harness makes a big difference vs a pure raw model.

> However AI cannot meaningfully handle feedback and learn

How do you think models are created? They are trained on feedback and learn.

Its not cheap but you can post train models. This is how custom models are mode, that deal with specific tasks more efficiently and accurately.

Example ... Composer? Its base Kimi v2.5 model that has been post-trained 2 weeks, to create Composer 2.5, what is a much better coding model.

Its literally trained to make less mistakes by feeding it correct data. Hell, a lot of the models you are using, are often the same base model, where v2.0 was the initial released model but the model keeps training, so when they release v2.1, its still the same model, but with more training time on feedback provided to v2.0.

LLM Models are not a cake you cook one time and they are done, and you start from zero again. If you have the money, and a powerful server setup, you can take a model like GLM 5.2 and post-train it, to reduce specific errors. Sure, you need a ton of money because its a large model.

But people have been doing this with 5M, 100M, 1B, 5B models for a long time already. To the point that some of the small models can do specific tasks, almost or better then some of the huge more general trained models.

That's all just prompting.

> How do you think models are created? They are trained on feedback and learn.

No one is post training models on a single mistake. At least I have not seen it. I also doubt it is effective. Post-training on a single failure will not meaningfully change the model. That even sidesteps the entire problem that you don't even have access to models if you use a provider like anthropic/openai

And telling someone not to repeat a mistake is… ?

Teaching modifies the learner. Prompting doesn't modify the model. It provides additional context that influences a single inference. A person who has learned something can apply it years later without being reminded. An LLM generally cannot unless the knowledge is incorporated into the model itself or provided again.

The value of LLMs is that they're stateless. With sufficiently detailed documentation and a well-bounded task, they are quite useful.

I'm not claiming LLMs are not useful, they most certainly are.

Take a human and prevent them from sleeping after covering a fact then years later they won’t know it.

>> Joe, how many times have I asked you to pay attention to commas. Come on, "let's eat grandma".

The whole "soul.md" stuff today is a poor approximation to that. But I wonder whether it will grow into it, like chain of thought prompting grew into reasoning models.

This is what I was referring to:

https://www.cnbc.com/2025/08/19/sam-altman-on-gpt-6-people-w...

It's marketing speak, but the goal is clearly there, no idea how achievable.

I feel like this line of thinking is kind of an unfair comparison. I'm not saying LLMs are magical beings that can suddenly learn by themselves after getting something wrong, but your "person making mistake then being corrected" assumes you do tell the person about the mistake and tell them to avoid doing the same mistake in the future, but for the "LLM making mistake" example you then intentionally avoid letting the prompt being changed in response to the mistake, which would be the "then being corrected" part on the LLM side of the comparison.

Similarly, if you just let a person make a mistake and don't let them know about the mistake, they might keep making that same mistake over and over again.

If you update how you use the LLM as you discover what mistakes it does, just like you'd correct a person, then you can use an LLM and also the LLM can "be taught to not make that mistake again".

> How many p are in strawperry?

> There are 0 “p”s in strawperry.

And I can trigger the same mistake with various words even when adjusting the prompt many times. So I cannot teach chatgpt to correctly count characters.

That'd be strange, what specific models?

Your example of "changing the prompt" doesn't actually tell the model what to avoid though, you're just rephrasing the question? Again you're not doing an equal comparison for the human, because in that example I bet you'd tell the human what they messed up and how to do it better, why not include that in the prompt when you compare these two?

Fable. (and other claude ones)

I have a clojure/babashka environment that I work in. My agents/claude files specifically explain this and how to work with it and what to use etc. Fable decided it didn't want to use bb/clj and ran python to do cli work in one project.

So I update the files in all projects to specify that python is strictly off limits/not to be used/never to run it/etc and created a preToolUse hook -- because I learned that just because an agents file says something doesn't mean it gets followed. Hence the hooks, fool me once ...

Later, I start work in another project with the hooks and updated files saying to use the established bb/cli tools and to absolutely never run/use python etc. What does it do on the first task? Tries to run python, gets hit with the preToolUse hook error and replies with something like -- oops my bad I tried to use python even though the instructions told me not too. Every once in awhile in new tasks I still see the hook error at some point but it doesn't apologize like it did before it just switches to proper tools after it can't run what it wants even though it's in the instructions and could have been avoided.

Why is it that almost every time someone makes a claim it turns out wrong when tested

please take a look at the error(s) made in the prior run. what could've been done better? create or modify an existing skill to emphasize this, or suggest additional language in AGENTS.md.

We’re not at the point where AI is capable of knowing what went wrong and self-aware enough to understand how it could reliably change its own behavior.

For months I’ve been trying to have the agents stop manually writing our auto-generated SQL migrations and run the command that generates them instead. SOTA models insist on occasionally getting it wrong.

Well this is the central bet of AI coding isn't it? We, the humans-in-the-loop, get better at knowing ahead of time which patterns AI will handle better than others, all the while the models actually get better.

You can do the same to an LLM with the same outcome.

People’s job is to partially reign in this volatile environment by creating processes with stable output.

[1] https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.p...

But your point stands: for critical business processes that need predictability, we indeed need determinism.

We're no different to AI. The code we write to solve a particular problem can (and probably does) change from day to day, depending on your "mood", what you had for breakfast, if you've been fighting with your significant other, other problems/human emotions.

You're there for blame, not much else. The systems are still going to churn garbage, but because it's a business, that business will rather pay less for garbage they can sell than pay you a living wage, eventuially.

Converging on a solution via an LLM is possible. The code, well, it wouldn't necessarily be the way a human would write it, better or not...

You mean the people creating actual business value in every industry where software isn’t the product? Speak not unless spoken to, Mr. cost-center!

In the same message you managed to twist my words, lack reading comprehension, jump to conclusions and do a personal attack.

Since you're new here, I recommend reading this: https://news.ycombinator.com/newsguidelines.html , especially this part: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

Please stop using this brand new hammer for glassmaking.

The AI companies' salespeople must be the greatest geniuses in the history of the world.

And I do more hand coding to guide the agent to useful patterns I want.

I feel like that’s how I used less capable agents a year ago. But I’m finding even with high quality agents, the slop creeps in.

I want more control. I want to save money. Hence going back to a more 80/20 agent to human LoC split.

GPT 5.5's double token cost was the threshold for me. These things are getting expensive quickly - the subsidized pricing can't go on forever.