Re:Andrej Karpathy talks about how Tesla's NNs are structure...
by londons_explore on Monday November 11, 2019 @05:46AM
(#21503695)
The duplication I see in the commercial world in ML inference hardware is in designs similar to the TPU... So a big ~128x28 accumulating mat-mul array, with enough memory throughput to get one operand in and out fast, and enough cache to store the other operand (weights) and switch between which weights are used so the mat-mul array can very efficiently do larger matrix sizes.
Also lookup tables for a bunch of activation functions.
That basic design can efficiently implement nearly any neural net architecture as long as the layer sizes are at least 128x128 and fixed point is okay.
The other exotic designs you suggested are more academic research things, and not yet deployed at scale in anyone's datacenters.
So which one design you suggest we all use for all our ML needs?