News for hackers. Stuff that matters.
Re:Kimi K2 Thinking, a SOTA open-source trillion-parameter r... by HarHarVeryFunny on Thursday November 06, 2025 @05:17PM (#45841189)
I think the question of what can be achieved with a small model comes down to what needs knowledge vs what needs experience. A small model can use tools like RAG if it is just missing knowledge, but it seems hard to avoid training/parameters where experience is needed - knowing how to perceive then act.

There is obviously also some amount (maybe a lot) of core knowledge and capability needed even to be able to ask the right questions and utilize the answers.

Re:Kimi K2 Thinking, a SOTA open-source trillion-parameter r... by lossolo on Friday November 07, 2025 @09:11AM (#45846580)
Small models handle simple, low context tasks most of the time correctly. But for more complex tasks, they fail due to insufficient training capacity and too few parameters to integrate the necessary relationships.