Re:Gemini 4 Neon
by weitendorf on Wednesday November 18, 2026 @05:32PM
(#50553263)
There's only a limited amount of hardware for training and serving models at scale, and which form an upper bound for what model size/inference can support.
A GB300 rack has 72x288GB=20TB VRAM. Do the math and 10T parameter models are just about the max size you can easily train and serve (at 1M context) without doing a lot of custom hardware development or limiting your scale.
A GB300 rack has 72x288GB=20TB VRAM. Do the math and 10T parameter models are just about the max size you can easily train and serve (at 1M context) without doing a lot of custom hardware development or limiting your scale.