Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models

Re:Inference-Aware Fine-Tuning for Best-of-N Sampling in Lar... by karmasimida on Monday April 28, 2025 @06:21AM (#43819600)

Isn't the BoN RL formulation similar to DeepSeek's GRPO algorithm? The latter seems to implicitly already captured this?

[ Parent | on HN ]

Re:Inference-Aware Fine-Tuning for Best-of-N Sampling in Lar... by Johnyhar on Monday April 28, 2025 @07:10AM (#43819918)

Wouldn't RL training, with the goal of aligning the LLM with the reward function R(x, y), result in the outputs of the trained LLM maximizing said reward function? How different are the rewards of the N outputs in BoN sampling, to justify its cost.

[ Parent | on HN ]