Re:A Look at Cerebras Wafer-Scale Engine: Half Square Foot S...
by 0-_-0 on Sunday November 17, 2019 @09:47AM
(#21558081)
> Or is there a reason that this won't work (or at least won't be worth it) for ANN structures?
If all the weights and activations fit in on-chip memory, then you can do calculations at close to 100% efficiency. If you want to simulate a 20k times larger network, you also need to transfer the 4 Billion parameters per iteration, which would take a singificantly longer time. In other words, you would be seriously bottlenecked.
Or is there a reason that this won't work (or at least won't be worth it) for ANN structures?