Re:Nvidia buying AI chip startup Groq for about $20B
by MobiusHorizons on Wednesday December 24, 2025 @08:18PM
(#46381219)
I am by no means an expert, but I think it is a process that allows training LLMs from other LLMs without needing as much compute or nearly as much data as training from scratch. I think this was the thing deepseek pioneered. Don’t quote me on any of that though.
Re:Nvidia buying AI chip startup Groq for about $20B
by tickerticker on Wednesday December 24, 2025 @11:56PM
(#46382168)
Yes. They bounced millions of queries off of ChatGPT to teach/form/train their DeepSeek model. This bot-like querying was the "distillation."
Re:Nvidia buying AI chip startup Groq for about $20B
by SirMaster on Thursday December 25, 2025 @01:16AM
(#46382542)
Why would OpenAI allow someone to do that?
Re:Nvidia buying AI chip startup Groq for about $20B
by MadnessASAP on Thursday December 25, 2025 @02:15AM
(#46382756)
They didn't, but how do you stop it? Presuming the scale that OpenAI is running at?
Re:Nvidia buying AI chip startup Groq for about $20B
by qcnguy on Friday December 26, 2025 @02:37PM
(#46395377)
They don't anymore. They introduced ID verification shortly after, but it's hard to stop completely while also scaling fast.
Re:Nvidia buying AI chip startup Groq for about $20B
by orbital-decay on Thursday December 25, 2025 @02:41AM
(#46382867)
They definitely didn't. They demonstrated their stuff long before OAI and the models were nothing like each other.
Re:Nvidia buying AI chip startup Groq for about $20B
by tensor on Thursday December 25, 2025 @12:25AM
(#46382303)
No, distillation is far older than deepseek. Deepseek was impressive because of algorithmic improvements that allowed them to train a model of that size with vastly less compute than anyone expected, even using distillation.
I also haven’t seen any hard data on how much they do use distillation like techniques. They for sure used a bunch of synthetic generated data to get better at reasoning, something that is now commonplace.
Re:Nvidia buying AI chip startup Groq for about $20B
by MobiusHorizons on Thursday December 25, 2025 @03:00AM
(#46382929)
Thanks it seems I conflated.
Could you elaborate? How is this done and what does this mean?