News for hackers. Stuff that matters.
Re:Gemini 4 Argon by mridulmalpani on Tuesday October 13, 2026 @12:59PM (#50074941)
Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race.

https://arena.ai/leaderboard/agent/chat/pareto

For almost same intelligence (+15.56%, +15.18%), drastic difference in cost/task ($3.51 vs $0.41)

Re:Gemini 4 Argon by vessenes on Tuesday October 13, 2026 @04:04PM (#50077835)
Muse reminded me we should have a time based chart as well - this year I hope we will begin to see time to completion stats at various intelligence tiers. The waiting makes a lot of things more tiring.
Re:Gemini 4 Argon by BrokenCogs on Tuesday October 13, 2026 @02:22PM (#50076239)
Every release I see a new benchmark with a new frontier. These graphs are just meaningless now
Re:Gemini 4 Argon by abeppu on Tuesday October 13, 2026 @02:29PM (#50076359)
In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace.
Re:Gemini 4 Argon by kridsdale1 on Tuesday October 13, 2026 @03:41PM (#50077493)
While true, Google genuinely has invested the most in computer and energy efficiency for LLMs.
Re:Gemini 4 Argon by LPisGood on Tuesday October 13, 2026 @11:03PM (#50081878)
This makes sense, but what evidence is there that this is true?
Re:Gemini 4 Argon by bootsmann on Wednesday October 14, 2026 @11:38AM (#50089378)
They run an LLM to summarize every search query for free and still turn more profit than ever?
Re:Gemini 4 Argon by LPisGood on Wednesday October 14, 2026 @11:51AM (#50089612)
That’s why it makes sense. It’s not evidence though. I was looking for investment numbers, papers, blog posts, etc.
Re:Gemini 4 Argon by yipinwong on Tuesday October 13, 2026 @02:29PM (#50076348)
you can give it a meaning if you see the relavivity between models. no need to disregard the stat at all.
Re:Gemini 4 Argon by ianm218 on Tuesday October 13, 2026 @03:26PM (#50077237)
Why do these benchmarks often show models running on Max i.e. Opus 5.5 (max) is on the frontier? I would think it is never really economical to run max
Re:Gemini 4 Argon by jonas21 on Tuesday October 13, 2026 @05:12PM (#50078843)
By definition, the model with the best performance will be on the pareto frontier (regardless of cost).
Re:Gemini 4 Argon by seizethecheese on Tuesday October 13, 2026 @06:11PM (#50079637)
No. Pareto frontier is cost performance tradeoff.
Re:Gemini 4 Argon by cloudbonsai on Tuesday October 13, 2026 @06:36PM (#50079891)
What jonas21 is stating is correct. If you have the absolute best performance, you are automatically on the Pareto frontier because you’re the most cost effective one (or the only model) on that performance tier.
Re:Gemini 4 Argon by seizethecheese on Tuesday October 13, 2026 @07:05PM (#50080209)
Huh, I guess you're right, but I interpreted jonas21's comment in relation to the above comment, which was about all models running on Max (which implies this would be a comment relating to all points on the pareto curve).
Re:Gemini 4 Argon by tibbar on Tuesday October 13, 2026 @09:59PM (#50081465)
It is true that the model with the overall best performance is on the Pareto frontier, but it is not necessarily true that a given model's best cost-performance tradeoff is at max reasoning. (IE you could have a model that is on the frontier at medium reasoning, but at max reasoning becomes more expensive and not sufficiently good enough to remain on the frontier.)
Re:Gemini 4 Argon by wordpad on Wednesday October 14, 2026 @12:02AM (#50082248)
If you're the best then you'd be thr first regarding of cost since there is nothing to compare to for that performance.
Re:Gemini 4 Argon by dfdydx on Wednesday October 14, 2026 @03:25AM (#50083716)
As other have stated, your point is relevant only for the single best performance point. The question is for interior points of the frontier where the tradeoff actually exists. For instance why is GPT 6 Luna (Max)on the frontier rather than GPT 6 Luna (Medium)? It is not because GPT 6 Luna (Max) is the most performant model regardless of cost.
Re:Gemini 4 Argon by casey2 on Wednesday October 14, 2026 @03:41AM (#50083859)
Why is this downvoted? From my understanding dominated solutions aren't on the paerto frontier e.g. in this case where argon and opus 5.5 are tied the lower cost argon should be the end of the curve.

From the graph it's a flat line, but looking at the point yields a negligible increase with no error bars (indeed if you look at fable 5.1 it's measured as lower than opus and argon). This just seems like bad data or blatant favoritism at worse.

Re:Gemini 4 Argon by serioussecurity on Wednesday October 14, 2026 @10:37AM (#50088409)
Yikes.
Re:Gemini 4 Argon by scotty79 on Wednesday October 14, 2026 @08:28AM (#50086639)
Check out slopalytics. You can draw a pareto line there with max reasoning excluded.
Re:Gemini 4 Argon by cheema33 on Tuesday October 13, 2026 @04:25PM (#50078167)
> Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race.

For chat, yes. But switch to code on the same page, and the picture changes. OpenAI takes the lead.

Re:Gemini 4 Argon by Silagi on Wednesday October 14, 2026 @08:41AM (#50086796)
Further, this benchmark looks to have done too poor a job of canvasing the model/reasoning level space to claim to represent the pareto frontier. Only running 6.1 sol and opus 5.5 on max seems absurd given that their other reasoning levels represent almost the entirety of the pareto frontier above luna for coding.