Re:Next Grok model training with 10T parameter model
by lifecodes on Wednesday April 08, 2026 @12:42PM
(#47692682)
I guess we are reaching the point where “10T parsmeters” sounds more like a marketing number than a meaningful metric.
Between moE, aggressive quantization, and synthetic data pipelines, it’s getting harder to tell whether bigger models are actually better, or just more expensive to train.
Would be more interesting to see ->
capability per dollar or per watt, not parameter count...
Between moE, aggressive quantization, and synthetic data pipelines, it’s getting harder to tell whether bigger models are actually better, or just more expensive to train.
Would be more interesting to see -> capability per dollar or per watt, not parameter count...