News for hackers. Stuff that matters.
Re:Qwen4-Max by lambda on Monday November 16, 2026 @01:08AM (#50514554)
They don't always run the benchmarks at every reasoning level for every model; only a few prominent ones. They do it for the US frontier labs, plus a few of the Chinese labs, but not consistently. You can see that DeepSeek has a few reasoning levels tested, GLM 5.3 has a couple, as does Qwen 3.8 27B, but not other Qwen models, and a lot of the Chinese models they only run at max reasoning.

It's possible that they will run Qwen 4 Max at other reasoning levels in the future, or it's possible that they never will. Regardless, it's unlikely to be better at any reasoning level on a price/performance basis than the GPT 6.1 Sol/GPT 6 Astra pareto frontier. It's close enough to Claude Opus 5.5 line at lower reasoning levels that it could potentially cross that if it doesn't loose too much performance at lower reasoning levels. But it's far enough from the global pareto frontier that it's unlikely that it would cross that at any resoning level.

Re:Qwen4-Max by esperent on Monday November 16, 2026 @01:21AM (#50514627)
[delayed]
Re:Qwen4-Max by brainwad on Monday November 16, 2026 @02:17AM (#50514946)
No, it's only valid to compare to models at the same cost level, regardless if that's their max level or not. Fable having higher effort modes than Qwen doesn't mean that you can't compare Fable medium to Qwen max.
Re:Qwen4-Max by esperent on Monday November 16, 2026 @03:30AM (#50515335)
[delayed]
Re:Qwen4-Max by aenis on Monday November 16, 2026 @02:56AM (#50515163)
> You can see for example that the cost per task for 6.1 sol is about 4x higher at max compared to medium thinking level.

It does not (universally) hold for long running workloads -- which is what people are increasingly doing.

Leave a Sonnet on medium, it will complete the task, spend more time and possibly significantly more token money, vs. Opus 5.5 on xhigh.