News for hackers. Stuff that matters.

A comment on Show HN: DistilKitPlus, a distillation framework between any LLMs.

Re:Show HN: DistilKitPlus, a distillation framework between ... by shikharM07 on Monday May 05, 2025 @12:21PM (#43896716)
this is kinda interesting but I'm curious what is the smallest model size that I can distill without compromising the accuracy?
[ Parent | on HN ]
Re:Show HN: DistilKitPlus, a distillation framework between ... by agokrani on Monday May 05, 2025 @01:33PM (#43897429)
We can distill 14B model to 4B model with performance improvements on AIME24 and GSM8K. We will share our results with a detailed blog post later.
[ Parent | on HN ]