News for hackers. Stuff that matters.
Re:Ask HN: Who uses open LLMs and coding assistants locally?... by ThatPlayer on Friday October 31, 2025 @06:36PM (#45777455)
With MoE models like gpt-oss, you can run some layers on the CPU (and some on GPU): https://github.com/ggml-org/llama.cpp/discussions/15396

Mentions 120b is runnable on 8GB VRAM too: "Note that even with just 8GB of VRAM, we can adjust the CPU layers so that we can run the large 120B model too"

There are no comments on this comment yet.