News for hackers. Stuff that matters.
Re:Unsloth GLM-5.2 – How to Run Locally by cogman10 on Monday June 22, 2026 @09:10PM (#48638845)
The newer models don't require more/better hardware. There's a small army of local llm enthusiasts who are running LLMs using 3090s and H100s because they have lots of memory. Them being old isn't really that big of an issue as the compute power needed is relatively low all things considered.

The number of parameters needed for these open weight models has mostly stabilized so the actual memory requirements aren't likely to change all that much.

Re:Unsloth GLM-5.2 – How to Run Locally by dannyw on Monday June 22, 2026 @11:27PM (#48639874)
Correct. The main bottleneck with LLM inference is, and have always been, memory bandwidth.

TPS = active weights in GB / your memory bandwidth.

That’s it for decode. That’s all.