A savant who confabulates about 10 % of the time, per the best benchmarks? How do you trust it?
How do you trust a human? Are humans infallible, or at least better than AI at explaining things? I known my answer.
I trust the Emacs contributors, the actual experts in the topic, to fact-check their own answers, because they are incentivized for doing so (reputation).
Re:Emacs in the Age of Magic
by orbital-decay on Thursday November 12, 2026 @04:22AM
(#50464904)
Have you ever worked with a modern agent? They let many weak assumptions slip because their cognitive functions aren't remotely as deep as human's, but they have also been trained to hyperfocus on verifying and grounding everything they do and say, you have to actively stop them from doing so when it's unnecessary. This not what what I can call confabulation in any good faith, let alone in 10% of cases. I think you've been misled by the old paper you cite below, it doesn't say what you likely think it says.
Yes, I mainly work with OpenCode, with GLM as an orchestrator agent and DeepSeek Flash as a worker agent. I’m assuming it’s modern? I don’t have the money to use either Fable or Astra, but my assumption is that the few more points they get on the benchmarks still don’t make up for the wild confabulated things they do in my projects.
I can't speak for the other providers but Gemini carbon feels like the difference between gpt3 and gpt5 when compared to Chinese open models.
Re:Emacs in the Age of Magic
by orbital-decay on Thursday November 12, 2026 @04:53AM
(#50465121)
Both are fine, though not frontier of course. They mostly take unwarranted shortcuts, trying to find the easiest way to reach the literal goal set by you, failing to see the broader picture. But they definitely do not confabulate, as in they don't just make up nonexistent things, they would rather walk extra mile to verify when they're not certain, which is the reason the software they make is trash but it works in the narrow sense. They can be rigid in their writing style do to mode collapse, but I haven't seen this leading to them inventing nonexistent functions or anything like that in a long while. RL makes them do all this, modern LLMs are very far from being next token predictors trained only on text.
> But they definitely do not confabulate, as in they don't just make up nonexistent things
I've definitely seen both Labs' "best" stuff confabulate to this day.
Agreed.
The harnesses / RLHF do a good job of prioritizing mechanisms to check results, which works out pretty well pragmatically for programming.
Nonetheless, I see Opus 5.5 spew out false information daily (never mind its code's unreadability).
I can't speak to Fable or Astra, because my employers, while threatening to fire us if we don't use the bullshit bots, have also refused to light the barrels of cash on fire that are required for those models.
As far as I can see, their fundamental nature has not changed, and truth is not something they have access to - just the called tool's exit code.
Folks can take EMacs as is and decipher it in isolation of Stallman, or any of the other authors.
But even then, how many people understand the internals of EMacs, or the interrelationships of the zillion lines of code in hundreds of files? Yet, millions of ignorant people leverage and apply EMacs to their daily tasks.
Look at what the REA impact is having on all sorts of code bases. The ONE THING that pushed Stallman over the edge and onto his path. The lack of source code to a print driver (or something along those lines).
Well, it’s all Open Source now.
Not only is the source “open”. It can be explained.
“Here is the source tree to EMacs. ELI5 the garbage collector.” “Working…”
It goes both ways. Now everyone has access to a savant to explain software, if that’s what they like.