Re:Cerebras’s giant chip will smash deep learning’s speed ba...
by tynpeddler on Thursday January 02, 2020 @03:42PM
(#21939555)
> Cerebras hasn’t released MLPerf results or any other independently verifiable apples-to-apples comparisons. Instead the company prefers to let customers try out the CS-1 using their own neural networks and data.
> This approach is not unusual, according to analysts. “Everybody runs their own models that they developed for their own business,” says Karl Freund, an AI analyst at Moor Insights. “That’s the only thing that matters to buyers.”
Sounds like instead of benchmarks, prospective customers get a chance to run a workload of their choice on the core before purchase. Assuming support is good, that's way better than looking at benchmarks, because you're guaranteed that the performances you're comparing are for workloads you care about.
Re:Cerebras’s giant chip will smash deep learning’s speed ba...
by Veedrac on Thursday January 02, 2020 @03:45PM
(#21939604)
The appropriately large models with public recognition I know of use attention, which is too memory-hungry to work effectively on the CS-1. The datasets aren't the issue.
I'm fine with skepticism. It's certainly plausible that they don't actually do all that well.
To me as a practitioner a meaningful metric would be "it trains an ImageNet classifier to e.g. 80% top1 in a minute". If it's not suitable for CNNs, do BERT or something else non-convolutional. Even better if I can replicate this result in a public cloud somewhere. They know this, and yet all we have is a single mention of a customer under an NDA and no public benchmarks of any kind, let alone any verifiable ones. If it did excel at those, we'd already know.