News for hackers. Stuff that matters.
Re:Kimi K2 Thinking, a SOTA open-source trillion-parameter r... by HarHarVeryFunny on Friday November 07, 2025 @01:15PM (#45849196)
> Not to nitpick words, but ablation is the practice of stripping out features of an algorithm ...

Ablation generally refers to removing parts of a system to see how it performs without them. In the context of an LLM it can refer to training data as well as the model itself. I'm not saying it'd be the most cost-effective method, but one could certainly try to create a small coding model by starting with a large one that performs well, and seeing what can be stripped out of the training data (obviously a lot!) without impacting the performance.

Re:Kimi K2 Thinking, a SOTA open-source trillion-parameter r... by oofbey on Friday November 07, 2025 @06:01PM (#45852297)
ML researchers will sometimes vary the size of the training data set to see what happens. It’s not common - except in scaling law research. But it’s never called “ablation”.