News for hackers. Stuff that matters.
Re:Gemini 4 Neon by seizethecheese on Wednesday November 18, 2026 @05:46PM (#50553443)
Okay, this "better at long horizon tasks" thing might explain why Carbon scores worse on most benchmarks yet is the larger model.

I looked into this using data I have from a battle system built on top of OpenCode (so we're trying to capture real world use). It looks like Carbon actually does a lot better when it uses significantly more output than opponents.

Win rate when output is less than 2.5x opponents: 53% (41/78)

Win rate when output is more than 2.5x opponents: 73% (41/56)

The p-val on this is 0.02. Data from https://agentduel.com/leaderboard.

There doesn't seem to be any relationship between actual size of task though. Win rate appears random based on total output of Carbon and also opponents. (Though our data is still small, so it might exist.)

Looks basically like Carbon was trained to think a lot longer, not necessarily that it's better on large tasks.

There are no comments on this comment yet.