Re:Magistral — the first reasoning model by Mistral AI
by redman25 on Wednesday June 11, 2025 @05:06AM
(#44245638)
It may not have been intentionally misleading. Some benchmarks can take a lot of horsepower and time to run. Their preparation for release likely was done well in advance of the model release before the new deepseek r1 model had even been available to test.
[ Parent
| on HN ]
Re:Magistral — the first reasoning model by Mistral AI
by reissbaker on Wednesday June 11, 2025 @08:18AM
(#44246826)
AIME24, etc are pretty cheap to run using any DeepSeek API. Regardless, they didn't even run the benchmarks for R1 themselves, they just republished DeepSeek's published numbers from January. They could have published the ones from May, but chose not to.