Re:TinyStories: How Small Can Language Models Be and Still S...
by a1o on Thursday January 02, 2025 @06:01PM
(#42579977)
I have no interest in that, I would like small models that I can integrate and run offline in software that I make it myself be IDEs or games. CLion has a nice predictive model for single line C++ completion that has 400 MBs.
[ Parent
| on HN ]
Re:TinyStories: How Small Can Language Models Be and Still S...
by refulgentis on Thursday January 02, 2025 @06:35PM
(#42580282)
Ah, totally possible, but wrapping llama.cpp will likely take a week to spike out and a month to stabilize across models.
The biggest problem for relying on it for local software is there's just too much latency for ex. game use cases currently. (among other UX bugaboos) (https://news.ycombinator.com/item?id=42561095)