Magistral — the first reasoning model by Mistral AI

Re:Magistral — the first reasoning model by Mistral AI by tootie on Tuesday June 10, 2025 @11:28AM (#44238057)

As an occasional user of Mistral, I find their model to give generally excellent results and pretty quickly. I think a lot of teams are now overly focused on winning the benchmarks while producing worse real results.

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by esafak on Tuesday June 10, 2025 @12:25PM (#44238657)

If so we need to fix the benchmarks.

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by tootie on Tuesday June 10, 2025 @04:07PM (#44240905)

I think there's a fundamental limit to benchmarks when it comes to real-world utility. The best option would be more like a user survey.

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by esafak on Tuesday June 10, 2025 @05:24PM (#44241611)

That's Chatbot Arena: https://lmarena.ai/leaderboard

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by jug on Tuesday June 10, 2025 @07:51PM (#44242788)

And unfortunately revealed to be largely a vibe check these days with that whole Llama 4 debacle. But why should we be surprised, really, when users have an easier time feeling if the replies sound human and conversational and _appear_ knowledgeable than actually outsmarting them. This Arena worked well in the ChatGPT 3.0 days… But now?

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by paulddraper on Tuesday June 10, 2025 @12:50PM (#44238915)

https://en.wikipedia.org/wiki/Goodhart%27s_law

[ Parent | on HN ]

Re:Magistral — the first reasoning model by Mistral AI by riku_iki on Tuesday June 10, 2025 @12:55PM (#44238971)

those who try to fix them are fighting alone against huge corps which try to abuse them..

[ Parent | on HN ]