News for hackers. Stuff that matters.
Re:DeepSeek-v3.2: Pushing the Frontier of Open Large Languag... by embedding-shape on Tuesday December 02, 2025 @12:44PM (#46124005)
> If you use MoE models (al modern >70B models are MoE), GPU utilization increases with batch size

Isn't that true for any LLM, MoE or not? In fact, doesn't that apply to most concepts within ML, as long as it's possible to do batching at all, you can scale it up and utilize more of the GPU, until you saturate some part of the process.

There are no comments on this comment yet.