Re:Cerebras Deploys Trillion-Parameter AI Models
by bubblethink on Sunday October 25, 2026 @12:22AM
(#50228296)
The reason 2D grids work well is because you can split tensor axes along whatever dim you want across PEs (batch, sequence, hidden, etc.). And your per PE code is relatively simple. It's of the type : get data from neighbor, compute my subtensor, send to neighbor. And you can implement broadcasts and reductions along any axis.
[ Parent
| on HN ]
Re:Cerebras Deploys Trillion-Parameter AI Models
by dnautics on Sunday October 25, 2026 @06:38AM
(#50230145)
You have to route around disabled subchips though.
Re:Cerebras Deploys Trillion-Parameter AI Models
by Nevermark on Sunday October 25, 2026 @11:32AM
(#50232660)
My guess is it is more like the opposite. They distribute spares evenly along every array in every dimension, so that there are always the same number of selected and unselected cores in each core-set.
With flexible addressing and bus-linking, along any set-line. So configuration happens the same way, with the same efficiency, regardless of whether any of the unselected elements are failed or not. As long as failures don't exceed the unselected-elements per element-array-set number.
And that number of unselected cores per element-array-set, is chosen to put wafer yields where they want them.
Trading a generous fixed wafer under-utilization for high yields, uniform configuration and processing efficiency, despite variable element failures.