Re:I'm making LLMs work on small and old devices
by NortySpock on Thursday November 12, 2026 @01:34PM
(#50471963)
Funny, I use that exact website to find passively cooled hardware to use as a server.
I did get Qwen3.5 0.8B working at 5 tokens per second on the iGPU of a Dell Wyse 7020. I mostly use it to complain to; I have not yet found an agentic task for it. But also the CPU does plenty of small services and terraria servers and such.
I did get Qwen3.5 0.8B working at 5 tokens per second on the iGPU of a Dell Wyse 7020. I mostly use it to complain to; I have not yet found an agentic task for it. But also the CPU does plenty of small services and terraria servers and such.
https://nortyspock.github.io/blog/running-a-language-model-o...