Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
by Garlef on Saturday August 01, 2026 @08:13AM
(#49133742)
If you grow your own tomatoes, you'll have free tomatoes!
(Doesn't really get you a BLT but hey... at least you'll have saved the world a bit because they're not from the supermarket)
/s
Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
by make3 on Saturday August 01, 2026 @03:33PM
(#49137633)
here at 0.5 tok/s, it's like getting 1/100 of a tomato for your lunch per hour, and you can't just let it grow, you have to look at it growing just as you're about to eat, and it doesn't accumulate between meals
Re:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
by rcxdude on Saturday August 01, 2026 @04:20PM
(#49138042)
It is worth thinking about, when you're buying something, how much of what you're paying is the supplier's margin. And whether they have efficiency advantages over you doing it yourself. But for large LLMs it does seem like there's a pretty big efficiency advantage to the rack-scale hardware in datacenters compared to hacks like this.
I hope that reasoning is an after-the-fact justification by the LLM that wrote this.
It's a ver interesting idea and I wouldn't mind trying it out, but with a smaller model. At 0.5t/s and reading many gigabytes from the SDD every second... I wonder if this wouldn't be extremely practical if targeting a 500gib or 250gib model, something that is still outside most consumers' laptop.