Cerebras’ new monster AI chip adds 1.4T transistors

Re:Cerebras’ new monster AI chip adds 1.4T transistors by frogblast on Thursday April 22, 2021 @06:58PM (#26909142)

Are there any good resources out there describing in practice how existing training workloads are distributed among GPUs? (using tensorflow, pytorch, or whatever else?).

I'm curious how the problem effectively gets sliced.

[ Parent | on HN ]

Re:Cerebras’ new monster AI chip adds 1.4T transistors by singhrac on Thursday April 22, 2021 @07:40PM (#26909457)

SOTA on the biggest language models (which is where effectively the largest models are) is here: https://www.microsoft.com/en-us/research/blog/zero-infinity-...

[ Parent | on HN ]