News for hackers. Stuff that matters.

A comment on Kimi Linear: An Expressive, Efficient Attention Architecture. Parent comment

Re:Kimi Linear: An Expressive, Efficient Attention Architecture by samus on Friday October 31, 2025 @09:21AM (#45771703)
We very much can, especially such a Mixture of Experts model with only 3B activated parameters.

With an RTX 3070 (7GB GRAB VRAM), 32 GB RAM and an SSD I can run such models at speeds tolerable for casual use.

[ Parent | on HN ]
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by embedding-shape on Friday October 31, 2025 @09:54AM (#45772009)
How many tok/s are you getting (with any runtime) with either the Kimi-Linear-Instruct or Kimi-Linear-Base on your RTX 3070?
[ Parent | on HN ]
Re:Kimi Linear: An Expressive, Efficient Attention Architecture by samus on Friday October 31, 2025 @04:09PM (#45776165)
With a Qwen3-32B-A3B (Q8) I'm getting 10-20 t/sec on KoboldAI, e.g., llama cpp. Faster than I can read, so good enough for hobby use. I expect this model to be significantly faster, but llama.cpp-based software probably doesn't support it yet.
[ Parent | on HN ]