For what it's worth, in our own metrics and benchmarks the intelligence is a pretty good match, but the cost per task is wildly out of whack. We see the open weight models like GLM-5.3-flash or Deepseek 4.1 Flash 1017 use a fraction of their tokens/cost of the benchmarks, which changes the cost-benefit equation a lot.
Re:Qwen4-Max
by anon373839 on Monday November 16, 2026 @02:19AM
(#50514962)
> Probably because it’s hard to benchmaxxed against.
It's mainly just a composite of existing benchmarks. It would be very easy to benchmax if, say, AA shared with a preferred lab which benchmarks it plans to upweight/downweight in the next version. Just speaking hypothetically, of course.
Re:Qwen4-Max
by puppymaster on Monday November 16, 2026 @02:33AM
(#50515034)
Hence the constant switching around and running the benchmark themselves. They have been pretty transparent about which benchmarks get added or removed with every release.
But admitting their model is weaker is a no go. So conspiracy theory here we go