Re:Artificial Analysis v5 and the benchmarker's dilemma
by artemisart on Friday October 30, 2026 @06:47AM
(#50302193)
Maybe you need to enable them in the filters, but AA is the best website I found with benchmarks numbers on all reasoning effort levels of recent frontier models.
Re:Artificial Analysis v5 and the benchmarker's dilemma
by lordofgibbons on Friday October 30, 2026 @06:56AM
(#50302253)
I dug into it more, and it seems like the reasoning efforts are available for the Intelligence Index, but not the Coding Agents benchmark I was interested in.
Re:Artificial Analysis v5 and the benchmarker's dilemma
by ac29 on Friday October 30, 2026 @06:47AM
(#50302189)
The other reasoning levels are almost always benchmarked, AA just doesnt show them in the default charts.
Re:Artificial Analysis v5 and the benchmarker's dilemma
by frenchtoast8 on Friday October 30, 2026 @07:40AM
(#50302610)
They don’t benchmark different reasoning levels for Deepseek models.
Re:Artificial Analysis v5 and the benchmarker's dilemma
by lordofgibbons on Friday October 30, 2026 @06:54AM
(#50302242)
I see them for GPT 6.1 Terra, but not Opus 5.5. I'm looking on the Coding Agents page.
Re:Artificial Analysis v5 and the benchmarker's dilemma
by kzrdude on Friday October 30, 2026 @12:09PM
(#50305961)
Is it different between major models how they scale with reasoning level? Some models where it hardly pays off? I expect it to be very similar across the board.
I get it, they can only benchmark so much. But I'd rather have a benchmark of how the model is actually used and offered to customers rather than the best case scenario for it.