Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware

Re:Show HN: Llmspeed – feel how fast a local LLM runs before... by lopotubez on Saturday October 17, 2026 @06:37AM (#50129803)

Models and inference engines are getting better and more optimized everyday so it is definitely worth experimenting with your GPU. You should probably check on newer finetuned and or REAP models Huggingface that could fit on your system.

I believe in your case the model quality matters a lot. To my (trust me bro) experience any MOE model, finetuned or not, with params < 19B even with Q4-quant always give lackluster results. So any Qwen2.5 wouldn't suffice to even simple coding tasks. For your case maybe you can bump it up to a Gemma4 E2B/E4B or Qwen3.5-9B. Also make sure that your current build of llamacpp actually uses your GPU with CUDA.

[ Parent | on HN ]

Re:Show HN: Llmspeed – feel how fast a local LLM runs before... by testerius on Saturday October 17, 2026 @07:01AM (#50129957)

Yes, I was using CUDA toolkit on my Debian. Anyway, thanks for reply. I think I could try something but definitely it looks like it will not have better quality, even for writing simple code function by function :P

[ Parent | on HN ]

Re:Show HN: Llmspeed – feel how fast a local LLM runs before... by zardo on Sunday October 18, 2026 @09:21AM (#50140877)

I haven't managed to run anything on my 8GB card that isn't blown away by free tier Gemini.

[ Parent | on HN ]