Re:Magistral — the first reasoning model by Mistral AI
by asadotzler on Tuesday June 10, 2025 @06:59PM
(#44242446)
The pre-training plateau is real. Nearly all the improvements since then have been around fine tuning and reinforcement learning, which can only get you so far. Without continued scaling in the base models, the hope of AGI is dead. You cannot reach AGI without making the pre-training model itself a whole lot better, with more or better data, both of which are in short supply.
[ Parent
| on HN ]
Re:Magistral — the first reasoning model by Mistral AI
by ethbr1 on Tuesday June 10, 2025 @07:32PM
(#44242657)
> with more or better data, both of which are in short supply
Hmmm. It's almost as if a company without a user data stream like OpenAI would be driven to release an end-user device for the sole purpose of capturing more training data...
Re:Magistral — the first reasoning model by Mistral AI
by MindTheAbstract on Wednesday June 11, 2025 @02:40AM
(#44244809)
While I tend to agree, I wonder if synthetic data might be reaching a new high with concepts like Google's AlphaEvolve. It doesn't cover everything, but at least in verifiable concepts, I could see it produce more valuable training data. It's a little unclear to me where AGI will come from (LLMs? EBMs - @LeCun)? Something completely different?)