Show HN:Getting GLM 5.2 running on my slow computer

Re:Show HN:Getting GLM 5.2 running on my slow computer by tarpitt on Thursday July 09, 2026 @06:45PM (#48853419)

I have epyc 9654 ES and a 7900 XTX. I was running the numbers, and even if I maxxed out the ram to like 12x32 gig sticks, it would cost me thousands more and I could only run GLM-5.2 at a couple tokens per second at q3. So this project is very promising because it suggests I could get pretty high speed and this CPU/motherboard combination suggests I have a lot of pci bandwidth that is unused.

I think another route might be looking at holding an even larger chunk of model weights in ram, and taking advantage of RAM<->GPU bandwidth, perhaps using a PCIe 5 GPU. This was my first thought since I have dedicated GPU.

If you are using Laptop, you're looking at shared memory between the iGPU and CPU. I've also tried that route, but I have always been skeptical of killing flash with too many reads, it essentially uses SSD like it's a consumable item.

I'm going to benchmark this right now with what I have and I'll get back to you on github.

[ Parent | on HN ]

Re:Show HN:Getting GLM 5.2 running on my slow computer by vforno on Thursday July 09, 2026 @06:54PM (#48853496)

Really thanks!!

[ Parent | on HN ]

Re:Show HN:Getting GLM 5.2 running on my slow computer by halcyonblue on Friday July 10, 2026 @01:13AM (#48855967)

At least for NVME, it is the write cycles that are limited. Read cycles are non-destructive and essentially unlimited.

[ Parent | on HN ]

Re:Show HN:Getting GLM 5.2 running on my slow computer by throwdbaaway on Friday July 10, 2026 @12:24PM (#48862081)

If you max out the ram, TG with q3 should be at least 10 t/s. And with dsa, it can still stay close to that number as the context grows.

[ Parent | on HN ]