I'm making LLMs work on small and old devices

Re:I'm making LLMs work on small and old devices by simedw on Thursday November 12, 2026 @01:54PM (#50472285)

Thanks for sharing! I'm exploring a similar direction, but optimising for extreme model size. I have gotten an LLM down to 970 KB, with some interesting results compared to GPT-2 (small).

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by mrashad74 on Thursday November 12, 2026 @01:55PM (#50472299)

This is extremely interesting, do you have a public repo or write up to share or not yet? if not yet please sign me up when you release it, Cheers!

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by simedw on Thursday November 12, 2026 @02:01PM (#50472416)

I'm planning to have a blog post up in the next two weeks or so (simedw.com), still have a lot of experiments to run, and training even tiny models takes a bunch of time.

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by mrashad74 on Thursday November 12, 2026 @02:39PM (#50473031)

following!

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by lilchungus on Thursday November 12, 2026 @02:00PM (#50472394)

Super interesting, not a CS/AI guy here, but I'm curious as to how you can train such a small-sized model give you as accurate an answer or good hypotheses to get you going

[ Parent | on HN ]