Supervised fine tuning on curated data is reinforcement learning

Re:Supervised fine tuning on curated data is reinforcement l... by anndvision on Tuesday July 29, 2025 @05:35PM (#44728526)

We recently ran similar experiments and saw that fine-tuning small models on automatically curated high-quality outputs from a large model can beat large-model performance while reducing inference costs by up to 30x and inference time by up to 4x.

We benchmarked closed-source (OpenAI, Google) and open-source (Qwen) models on multi-turn maze navigation (BabyAI), agentic RAG (Multi-Hop), and agentic tool use (τ-bench).

We're still running a few experiments and plan to update the post with additional results in a few days.

Looking forward to trying out importance weighting soon!

Curated Behavior Cloning: Small LLMs Can Beat Large Ones at 5-30x Lower Cost: https://www.tensorzero.com/blog/curated-behavior-cloning-sma...

[ Parent | on HN ]

Re:Supervised fine tuning on curated data is reinforcement l... by chongliqin on Tuesday July 29, 2025 @06:05PM (#44728800)

Cool! If you are interested, we have open sourced our code: https://github.com/emmyqin/iw_sft

[ Parent | on HN ]

Re:Supervised fine tuning on curated data is reinforcement l... by anndvision on Tuesday July 29, 2025 @07:05PM (#44729277)

thanks

[ Parent | on HN ]

Re:Supervised fine tuning on curated data is reinforcement l... by TheTaytay on Tuesday July 29, 2025 @10:46PM (#44730509)

Thanks for this - I’ve spent the last hour reading your docs and blog. I like the primitives you’ve exposed in your APO, and particularly like the decision to separate out the structured inputs from the prompt when you record an LLM call, so I can finally perform optimizations and evals on past calls.

Quick question : you mentioned unsloth in the blog post. Which of the fine tuning providers mentioned is using unsloth under the hood?

[ Parent | on HN ]

Re:Supervised fine tuning on curated data is reinforcement l... by GabrielBianconi on Tuesday July 29, 2025 @11:07PM (#44730602)

[I'm his coworker.] We ran Unsloth ourselves on a GPU-by-the-hour server. We have a notebook in the repository showing how to query historical data and use it with Unsloth.

It's a WIP PR that we plan to merge soon: https://github.com/tensorzero/tensorzero/pull/2273

[ Parent | on HN ]