Re:DeepSeek-V4-Flash Update
by teravor on Friday July 31, 2026 @11:00PM
(#49130658)
a specialist model sufficiently post-trained can outperform a frontier model while being dirt cheap.
what we did was distill GLM 5.2 into a 27B model on SQL and then post-train it with RL afterward. the result outperformed even Fable on that one task. the distillation step is just good sense in this workflow, to bootstrap a smaller model to the utmost you can before actually doing RL.
Re:DeepSeek-V4-Flash Update
by MrDrMcCoy on Saturday August 01, 2026 @11:31AM
(#49135297)
Do you have any reference guides on how to perform this distillation? I have a use case for this and hardware to run it on, but have never seen how it's done.
what we did was distill GLM 5.2 into a 27B model on SQL and then post-train it with RL afterward. the result outperformed even Fable on that one task. the distillation step is just good sense in this workflow, to bootstrap a smaller model to the utmost you can before actually doing RL.