News for hackers. Stuff that matters.
Re:Unsloth GLM-5.2 – How to Run Locally by Tepix on Tuesday June 23, 2026 @09:42AM (#48644869)
$50K seems low if you want to run, say, GLM 5.2 4bit fast enough for a team for devs.

You need something like 6x RTX Pro 6000 at $11800 each plus a nice server (add $10000) = $80800 and then quite a bit of electricity.

Re:Unsloth GLM-5.2 – How to Run Locally by theYipster on Tuesday June 23, 2026 @01:43PM (#48648529)
You don't need all of the model in VRAM. 1 or 2 RTX Pro 6000s will do. $50K will get you there very nicely, and on a 1600 watt PSU if you go for the MAX-Q versions. (The same wattage PSU I'm typing this on, and have been using over the last 5 years.)
Re:Unsloth GLM-5.2 – How to Run Locally by Tepix on Wednesday June 24, 2026 @02:23AM (#48655904)
If you want decent performance (more than say 20 tokens/s) for your dev team, you absolutely do need all of the model in VRAM.