News for hackers. Stuff that matters.
Re:DeepSeek-v3.1-Terminus by Jgoauh on Monday September 22, 2025 @10:10AM (#45333716)
Re:DeepSeek-v3.1-Terminus by JimDugan on Monday September 22, 2025 @11:14AM (#45334600)
Dumb collation of benchmarks that the big labs are essentially training on. Livebench.ai is the industry standard - non contaminated, new questions every few months.
Re:DeepSeek-v3.1-Terminus by IgorPartola on Monday September 22, 2025 @11:31AM (#45334853)
Thanks! Are the scores in some way linear here? As in, if model A is rated at 25 and model B at 50, does that mean I will have half the mistakes with model B? Get answers that are 2x more accurate? Or is it subjective?
Re:DeepSeek-v3.1-Terminus by esafak on Monday September 22, 2025 @07:25PM (#45340887)
I believe the score represents the fraction of correct answers, so yes.
Re:DeepSeek-v3.1-Terminus by alexeiz on Tuesday September 23, 2025 @12:06PM (#45348957)
It says the best "coding index" is held by Grok 4 and Gemini 2.5 Pro. Give me a break. Nobody uses those models for serious coding. It's dominated by Sonnet 4/Opus 4.1 and GPT-5.