News for hackers. Stuff that matters.
Re:Gemini 4 Neon by jorl17 on Wednesday November 18, 2026 @05:00PM (#50552816)
Here's my average experience with Carbon.

(After a long conversation with existing messages)

User Prompt: Ok, so SSH to the unraid server and update the plex db

Reply:

I need to think how to answer this.

The user is asking me to SSH into the unraid server. This seems to imply there is a machine to which we can SSH to. I should look for places where hosts may exist.

The implication is that once SSH is established to unraid, the plex db can be updated. It seems plex is a service that the machine has in which we can run updates.

I'm wondering if plex might be the Plex service the user has been discussing.

So the user doesn't want to SSH to the server - actually they want to SSH to the server.

... Formulating a plan ...

[some messages later]

Aha! THIS IS THE SMOKING GUN! `~/.ssh/config` HAS THE REQUESTED HOST!!! IT'S TIME TO CONNECT!!

[some messages later]

Ok, time to act. I'll ssh to the server and follow coherent rules.

----

I am shocked at how primitive its reasoning sounds (it reads just like gemini 2.5-pro's old reasoning). It often gets the job done (although it's quite slow and can't tackle more sophisticated problems).

Perhaps Anthropic's internal reasoning is also like this, but they really rewrite it to make it seem more coherent (which certainly sells it on me)? I almost want to click stop just by reading this nonsense.

Re:Gemini 4 Neon by aenis on Wednesday November 18, 2026 @05:21PM (#50553097)
One of my long horizon tasks benchmarks involves writing logic controlling a team of soldiers in a simulated environment. Argon and Carbon both made such discoveries such as "it would seem that i need to be pointing at an enemy to shoot them". Its almost entertaining to read, its like its discovering the world on first principles.

(And it did produce a very competent algorithm, though it took a few days and a metric ton of tokens; I just triggered the same task with Neon).

Re:Gemini 4 Neon by jorl17 on Wednesday November 18, 2026 @05:30PM (#50553233)
I'm curious about how it goes with Neon! This night I'll leave Neon with some lofty goals on a project and I'll see how it fares too.
Re:Gemini 4 Neon by aatd86 on Wednesday November 18, 2026 @06:49PM (#50554012)
lol Aenis the god.
Re:Gemini 4 Neon by iamdelirium on Wednesday November 18, 2026 @05:23PM (#50553142)
All displayed reasoning is not real reasoning for all models now. It's fake since they can be trained on.
Re:Gemini 4 Neon by jorl17 on Wednesday November 18, 2026 @05:29PM (#50553227)
I want to believe you, and it's what makes the most sense! But if that's the case, why is google (intentionally?) presenting such terrible reasoning traces? To make sure that if they're trained on, they're trained on garbage? I guess it makes sense, but still...

I was not really fabricating the example I gave too much. Here's a verbatim transcript from a session which was filled with this stuff:

    ... thought LET'S READ RecordReader.cs RIGHT NOW!! RecordReader.cs IS WHERE EVERY SINGLE TABLE, ReadPlan, DecideValueVisibility, ReadRowCounts , AND ShowValues READS ITS DATA!! Let's read RecordReader.cs (lines 1-300)!
Re:Gemini 4 Neon by 8note on Wednesday November 18, 2026 @05:38PM (#50553329)
reasoning traces also arent real reaaoning. its what tokens warm up the outpt the best
Re:Gemini 4 Neon by dist-epoch on Wednesday November 18, 2026 @05:46PM (#50553438)
What you're seeing is a small model summarizing and possibly obfuscating the real thinking.
Re:Gemini 4 Neon by jorl17 on Wednesday November 18, 2026 @05:49PM (#50553474)
Yes, I understand it.

It seems rather ironic that Anthropic givens the most logical reasoning traces, though. You'd think if they're ahead they'd either hide them or make them produce garbage (like what seems to come out of Google's models).

Re:Gemini 4 Neon by orbital-decay on Wednesday November 18, 2026 @06:08PM (#50553657)
Neither has any relation to the actual reasoning trace. The summary is actively misleading and you'd be surprised how different it is from the raw CoT of the model. Just ignore it.
Re:Gemini 4 Neon by jorl17 on Wednesday November 18, 2026 @06:13PM (#50553700)
This makes sense. Nonetheless, I find that reading Anthropic's reasoning traces is sometimes a trigger for researching more stuff. Often I'll see models thinking (or fake thinking, whatever) about something and I'll be curious to check out more about that topic if I'm not particularly familiar with it.
Re:Gemini 4 Neon by jrowen on Thursday November 19, 2026 @01:10AM (#50556619)
FWIW, you're not wrong. As I said in my other comment, I don't really know what other models are up to but Claude is definitely demonstrating a deep level of understanding, and I learn things from it all the time.

Sometimes I'm like...is everybody else using the same thing that I am? This is insane.

Re:Gemini 4 Neon by jorl17 on Thursday November 19, 2026 @01:22AM (#50556684)
That's the exact same feeling I have.

It's obvious that what we're getting are summaries. Sometimes I think I even see vestiges of the summarizing model instructions. But, that being the case, it's bizarre why Gemini's summarizer acts like a toddler, and Claude's actually lets me learn and, more importantly, stop it at its tracks when I see it's going down the wrong path.

On the other end of the spectrum, if you toy around with the open-weights models, you see the insanity that is going on in their output. I can totally believe in a scenario where claude is basically doing that and then summarizing it into something coherent, getting rid of the 10 opposing viewpoints, "aha!" and "Hmmm, but wait!", but it does it elegantly.

I've tried constantly to use other models (and I do), but Claude spoils me to no end. I keep thinking I must be holding everything wrong.

Re:Gemini 4 Neon by deaux on Wednesday November 18, 2026 @06:58PM (#50554095)
Off-topic, but this kind of conversation tells me just how little the average HNer still uses LLMs outside of specific coding tasks, or at least pays attention to them. Which in turn explains so many comments. Probably a good thing though.

You're of course completely right. Interestingly the "summaries" have only become more diverged over time. I now believe the big 3 are barely summarizing at all any more. I deem it likely that the small models don't get the CoT at all, they just completely make up something "plausible" solely based on the user message with a little bit of previous context. Much of the time it will be the complete opposite of something I asked for, or barely related.

Your suggestion is the right one. Everyone who uses LLMs for their job should spend some time with open models like Deepseek and see what the thinking actually looks like.

Re:Gemini 4 Neon by jrowen on Thursday November 19, 2026 @01:05AM (#50556578)
I mean, it does though. The sequence of thoughts that Claude goes through in response to my queries blew me away the first time I used it, and it continues to impress. Just the act of putting those sentences together demonstrates such a deep understanding of what I'm saying. If it's just an approximation, whatever it's actually doing is even better.

I haven't even really tried to use other models in the same way, and this thread makes me feel like I don't need to.

Re:Gemini 4 Neon by behnamoh on Wednesday November 18, 2026 @05:59PM (#50553573)
Not only that, Anthropic injects fakes reasoning and tool calls in there to make it harder for distillers. It's shitty behavior but you'd do the same if you had an edge you wanted to maintain.