News for hackers. Stuff that matters.

A comment on Magistral — the first reasoning model by Mistral AI. Parent comment

Re:Magistral — the first reasoning model by Mistral AI by danielhanchen on Tuesday June 10, 2025 @11:00AM (#44237646)
Their paper https://mistral.ai/static/research/magistral.pdf is also cool! They edited GRPO via:

1. Removed KL Divergence

2. Normalize by total length (Dr. GRPO style)

3. Minibatch normalization for advantages

4. Relaxing trust region

[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by Onavo on Tuesday June 10, 2025 @11:26AM (#44238026)
> Removed KL Divergence

Wait, how are they computing the loss?

[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by danielhanchen on Tuesday June 10, 2025 @11:32AM (#44238091)
Oh it's the KL term sorry - beta * KL ie they set beta to 0.

The goal of it was to "force" the model not to stray to far away from the original checkpoint, but it can hinder the model from learning new things

[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by mjburgess on Tuesday June 10, 2025 @11:44AM (#44238218)
It's just a penalty term that they delete
[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by trc001 on Tuesday June 10, 2025 @06:37PM (#44242276)
It's become trendy to delete it. I say trendy because many papers delete it without offering any proof that it is meaningless
[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by gyrovagueGeist on Tuesday June 10, 2025 @01:58PM (#44239510)
Does anyone know why they added minibatch advantage normalization (or when it can be useful)?

The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test problems, and (mean-of-minibatch)-normalization doesn't seem theoretically motivated for convergence to the optimal policy?

[ Parent | on HN ]
Re:Magistral — the first reasoning model by Mistral AI by danielhanchen on Tuesday June 10, 2025 @07:05PM (#44242496)
Tbh I'm unsure as well I took a skim of the paper so if I find anything I'll post it here!
[ Parent | on HN ]