News for hackers. Stuff that matters.
Re:Kimi K3.1 Pro weights out by reissbaker on Wednesday November 04, 2026 @01:31PM (#50369635)
If your benchmark puts DS4.1-Flash at the same tier as K3 or K3.1, I'm sorry to say your benchmark is broken. It really isn't, as much as I wish it was.
Re:Kimi K3.1 Pro weights out by seizethecheese on Wednesday November 04, 2026 @01:33PM (#50369641)
We allow early voting. This is against what most arenas do, but is on purpose. If the model is faster to do simple stuff, it should be chosen. In this sense, DS4.1-Flash is better on many tasks.

You can remove early voting from leaderboard and they are no longer the same.

Re:Kimi K3.1 Pro weights out by nostrebored on Wednesday November 04, 2026 @06:05PM (#50372696)
But how do you verify a fair or representative distribution of complexity?
Re:Kimi K3.1 Pro weights out by seizethecheese on Wednesday November 04, 2026 @11:23PM (#50375155)
We validate people are doing real work using LLM judges.
Re:Kimi K3.1 Pro weights out by purerandomness on Wednesday November 04, 2026 @07:06PM (#50373200)
At the point where I choose an expensive model like Kimi 3.1, speed doesn't matter, at all.

Usually, it's because all else failed, and I have to resort to burning Tokens with more expensive/better models, so a few minutes here and there really should not matter.

Re:Kimi K3.1 Pro weights out by seizethecheese on Wednesday November 04, 2026 @11:24PM (#50375162)
[delayed]
Re:Kimi K3.1 Pro weights out by rocho on Thursday November 05, 2026 @03:59AM (#50376873)
But his point is that the faster models could _not_ complete the task satisfactorily.

I've seen this myself too: DeepSeek 4.1 Flash is really good and incredibly cheap at constrained implementations, searches, etc. However, frequently the implementations don't take into account all possible cases they should cover. Then I'm either forced to step in and steer it very precisely, or have a heavier model review and provide detailed feedback. I've had good success with this mixed workflow using GPT 6.1 Sol as reviewer. Then my work focuses mostly at the beginning (architecture, goals, constraints), and at the end (final review or very quick intermediate reviews). I'm not aware of benchmarks that simulate this kind of mixed workflow with different models.

Re:Kimi K3.1 Pro weights out by haizhung on Thursday November 05, 2026 @02:17AM (#50376140)
I tried both of them for agentic coding and didn’t feel like K3 was adding so much benefit to justify its almost 10x price (on the modem provider I’m on).

In fact, I barely see a difference between the results at all. The models being tied smells correct to me.

I didn’t try 3.1 yet, though.