News for hackers. Stuff that matters.
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by IshKebab on Tuesday March 28, 2023 @05:58PM (#35348544)
It's a pretty mad architecture tbh. Compile times must be absolutely insane. Also Tesla's Dojo also uses a manufacturing technique that has basically obsoleted their WSI design already.
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by anon291 on Tuesday March 28, 2023 @08:20PM (#35350171)
Compile times are not a whole lot different than any other large model build. It's a kernel based compilation pipeline and the kernels are simply tiled over a 'core' area in the weight streaming architecture.
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by IshKebab on Wednesday March 29, 2023 @04:11AM (#35354090)
I seriously doubt that. What's your source?

I used to work for a competitor with a more flexible architecture and even our compile times were bad (significant fractions of a day in some cases). And we didn't have to do place and route!

I just googled it and it's apparently bad enough that they had to implement incremental place and route.

Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by anon291 on Wednesday March 29, 2023 @10:33PM (#35367134)
I used to work at Cerebras. They don't do place and route anymore. That's the old pipeline mode. They've shifted to weight streaming