The Kimi K3 Moment

Re:The Kimi K3 Moment by jrflo on Saturday July 18, 2026 @06:32PM (#48963114)

We really need to stop using $/M tokens as the pricing benchmark. I've found that the number of tokens used tends to be a bigger factor than the listed per token price. The cost per task vs. intelligence curve is really what you care about, and in my estimation Chinese models are just not there. They are focused on benchmaxing and getting the highest raw score they can, rather than efficiency.

[ Parent | on HN ]

Re:The Kimi K3 Moment by resonious on Saturday July 18, 2026 @07:46PM (#48963589)

The artificialanalysis cost per task chart has DeepSeek as the clear winner and Fable as the clear loser. But I would still pick Fable for some tasks, so that also can't be all there is to it.

But I agree that price per token figure is not great. It seems even the tokens per character can vary between models, so it's basically useless.

[ Parent | on HN ]

[dead]

Re:The Kimi K3 Moment by SwellJoe on Sunday July 19, 2026 @04:12PM (#48971320)

In my benchmarks of security auditing abilities of models, DeepSeek was roughly an order of magnitude cheaper than either GPT 5.5 or Opus 4.8, less than ten cents per task vs. roughly a buck each for the best American models at the time.

GPT 5.5 Pro was ~230x at almost $23 per task.

DeepSeek is my go-to when I need an API, and local Gemma 4 won't do because it's either too slow or not capable enough. DeepSeek isn't at the frontier but it's good enough for a lot of things, very cheap, and quite fast. Flash is even faster and cheaper, and still better than anything I can host locally.

[ Parent | on HN ]

Re:The Kimi K3 Moment by MrBuddyCasino on Sunday July 19, 2026 @08:06AM (#48967396)

Almost as if centuries of passing the imperial exam by either skill or cheating influenced Chinese culture a lot.

[ Parent | on HN ]

Re:The Kimi K3 Moment by RayVR on Saturday July 18, 2026 @08:56PM (#48963991)

Yes, this is already accounted for in many benchmarks, but without deep context of the problem type, the top line pricing is the best starting point.

In my own experience, Fable is more token efficient than opus 4.8 with a higher likelihood of completing tasks correctly or at least with minimal corrective work. Opus regularly struggled to gather the correct context and reason effectively about what it had gathered.

GPT-5.6-sol crushes fable in speed and token efficiency and is clearly superior across many tasks that matter for me.

I also find all models from anthropic after opus 4.6 to suffer from the same ai slop language that long plagued OpenAI and seems to have been reduced drastically in 5.6

[ Parent | on HN ]

Re:The Kimi K3 Moment by foobar10000 on Saturday July 18, 2026 @09:56PM (#48964329)

Don't forget that you are not really seeing the thinking tokens used - so non-trivial to count them.

[ Parent | on HN ]

Re:The Kimi K3 Moment by progx on Sunday July 19, 2026 @04:39AM (#48966084)

Ockbench: https://ockbench.github.io/ show token usage and accuracy, but you have to compare it with other stats (prices and speed).

[ Parent | on HN ]

Re:The Kimi K3 Moment by fallingbananna on Sunday July 19, 2026 @08:14AM (#48967446)

While I do agree that cost per task is what customers should care about, and not cost per token. Cost per token is an objective metric. Cost to do a task can vary a lot. Different tasks, different prompts, or just pure randomness nature of models make it a bit harder to define this as an objective metric.

Unless the cost per token is prohobitedly high, people can often try the model out themselves and make a subjective judgement of how effective and efficient is it at solving tasks they usually deal with, using their setup.

[ Parent | on HN ]