Re:DeepSeek-v3.1-Terminus
by IgorPartola on Monday September 22, 2025 @08:54AM
(#45332751)
I wish there was some easy resource to keep up with the latest models. The best I have come up with so far is asking one model to research the others. Realistically I want to know latest versions, best use case, performance (in terms of speed) relative to some baseline, and hardware requirements to run it.
[ Parent
| on HN ]
Re:DeepSeek-v3.1-Terminus
by __mharrison__ on Monday September 22, 2025 @12:09PM
(#45335468)
I use Aider heavily and find their benchmark to be pretty good. It is updated relatively frequently (a month ago, which may be an eternity in AI time).
have you tried
https://artificialanalysis.ai/
Dumb collation of benchmarks that the big labs are essentially training on. Livebench.ai is the industry standard - non contaminated, new questions every few months.
Re:DeepSeek-v3.1-Terminus
by IgorPartola on Monday September 22, 2025 @11:31AM
(#45334853)
Thanks! Are the scores in some way linear here? As in, if model A is rated at 25 and model B at 50, does that mean I will have half the mistakes with model B? Get answers that are 2x more accurate? Or is it subjective?
I believe the score represents the fraction of correct answers, so yes.
It says the best "coding index" is held by Grok 4 and Gemini 2.5 Pro. Give me a break. Nobody uses those models for serious coding. It's dominated by Sonnet 4/Opus 4.1 and GPT-5.
> asking one model to research the others.
that's basically choosing are random with extra steps!
Re:DeepSeek-v3.1-Terminus
by throwup238 on Monday September 22, 2025 @09:54AM
(#45333527)
Research not spit out the answer based on weights. Just ask Gemini/Claude to do deep research on /r/LocalLLama and HN posts.