Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ...
by IshKebab on Tuesday March 28, 2023 @05:58PM
(#35348544)
It's a pretty mad architecture tbh. Compile times must be absolutely insane. Also Tesla's Dojo also uses a manufacturing technique that has basically obsoleted their WSI design already.
[ Parent
| on HN ]
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ...
by anon291 on Tuesday March 28, 2023 @08:20PM
(#35350171)
Compile times are not a whole lot different than any other large model build. It's a kernel based compilation pipeline and the kernels are simply tiled over a 'core' area in the weight streaming architecture.
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ...
by IshKebab on Wednesday March 29, 2023 @04:11AM
(#35354090)
I seriously doubt that. What's your source?
I used to work for a competitor with a more flexible architecture and even our compile times were bad (significant fractions of a day in some cases). And we didn't have to do place and route!
I just googled it and it's apparently bad enough that they had to implement incremental place and route.