News for hackers. Stuff that matters.
Re:AI Came for the Coders. Understanding Why Tells Us Who's ... by wolttam on Friday November 20, 2026 @09:25AM (#50575071)
I almost never hit the failure modes of LLMs anymore, they have been getting much more reliable year over year.

More and more it would appear the failure modes are a training data issue

Re:AI Came for the Coders. Understanding Why Tells Us Who's ... by geraneum on Friday November 20, 2026 @10:31AM (#50576175)
> I almost never hit the failure modes of LLMs anymore

Of course you do. Just because you don’t see the LLM trying the ls command for the billionth time with the wrong arguments, just to be corrected by the harness, doesn’t mean it doesn’t happen. Of course I use the ls cli as an example. Could be any task. That’s what I mentioned about the harness and that’s why they shine in the fields with hard verification mechanisms and feedback loops.

Re:AI Came for the Coders. Understanding Why Tells Us Who's ... by wolttam on Friday November 20, 2026 @01:17PM (#50579324)
I actually use my own harness that doesn't attempt to fix broke tool calls. Yes models are not perfect and they do make mistakes, but my experience has been that reliability has been trending only upward. I really can't remember the last time I saw a tool call loop.

(this has been my experience since ~DSv4 from April)

Re:AI Came for the Coders. Understanding Why Tells Us Who's ... by geraneum on Friday November 20, 2026 @02:07PM (#50580144)
> I actually use my own harness that doesn't attempt to fix broke tool calls.

It’s not the harness that attempts to fix the broken tool calls. It’s the LLM.

> last time I saw a tool call loop

We are not talking about tool call loops

Re:AI Came for the Coders. Understanding Why Tells Us Who's ... by wolttam on Friday November 20, 2026 @04:11PM (#50582166)
> It’s not the harness that attempts to fix the broken tool calls. It’s the LLM.

Then what did "Just to be corrected by the harness" mean?

> last time I saw a tool call loop

Reasoning loops, output loops, etc, I just don't tend to see them with the models I use.

All I'm saying is that I find LLMs to have been consistently getting more reliable, to the point that their failure modes don't really stand out.