News for hackers. Stuff that matters.
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs by butILoveLife on Wednesday March 11, 2026 @09:24AM (#47335255)
>. I run quantized 70B models locally (M2 Max 96GB, llama.cpp + LiteLLM), and memory bandwidth is always the bottleneck.

I imagine you got 96gb because you thought you'd be running models locally? Did you not know the phrase Unified Memory is marketing speak?

There are no comments on this comment yet.