Show HN: DistilKitPlus, a distillation framework between any LLMs

Posted by ayushnangia16 on Monday May 05, 2025 @12:12PM
from the show-hn dept.

https://github.com/agokrani/distillKitPlus

Over the past few months, I have built a distillation toolkit that supports cross-tokenizer distillation (e.g., distilling from LLaMA to Qwen vocab, or others). This approach has worked well on reasoning datasets like AIME, and we’ve validated on models like Phi and Qwen.

We’ve also integrated Modal for quick deployment (with $30/month credits to try it out).

Would love any feedback!

GitHub: https://github.com/agokrani/distillKitPlus

Docs: https://distillkitplus.mintlify.app/

( 4 comments | 2 points | on HN )

Re:Show HN: DistilKitPlus, a distillation framework between ... by shikharM07 on Monday May 05, 2025 @12:21PM (#43896716)

this is kinda interesting but I'm curious what is the smallest model size that I can distill without compromising the accuracy?

[ Parent | on HN ]

Re:Show HN: DistilKitPlus, a distillation framework between ... by agokrani on Monday May 05, 2025 @01:33PM (#43897429)

We can distill 14B model to 4B model with performance improvements on AIME24 and GSM8K. We will share our results with a detailed blog post later.

[ Parent | on HN ]

Re:Show HN: DistilKitPlus, a distillation framework between ... by vijit-singh on Monday May 05, 2025 @02:58PM (#43898265)

this is very cool. will try it out.

[ Parent | on HN ]

Re:Show HN: DistilKitPlus, a distillation framework between ... by vikramxD on Monday May 05, 2025 @03:31PM (#43898590)

Cool , are you accepting contributions for adding new models

[ Parent | on HN ]