News for hackers. Stuff that matters.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by modeless on Sunday November 10, 2019 @05:43AM (#21497435)
Google has Edge TPUs for use outside datacenters, and they don't support training. Neither do the chips Tesla made for their cars. It's a pretty different problem.
Re:Andrej Karpathy talks about how Tesla's NNs are structure... by antpls on Sunday November 10, 2019 @05:48AM (#21497452)
I wouldn't be so sure. Edge TPUs could be the exact same architecture than Google Cloud TPUs, but as you need less computation power for inferring than training, they have simply less transistors on the die and could be underclocked.

In other words, Cloud TPUs could be the same architecture than Edge TPUs but scaled to an higher frequency and more packed.

I guess we need sources to confirm.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by londons_explore on Sunday November 10, 2019 @08:44AM (#21497920)
Training is currently done in floating point math, whereas inference can be done fixed point without much loss of performance. Fixed point is ~10x cheaper in terms of power and silicon area for equal performance.

Also, training requires a lot more RAM per unit of compute, since it needs to store all past layer activations, whereas for inference, that is unnecessary.

As far as I know, no player who has developed dedicated ML hardware (as opposed to using GPU's) uses the same hardware for both inference and training.

Re:Andrej Karpathy talks about how Tesla's NNs are structure... by modeless on Sunday November 10, 2019 @06:10AM (#21497513)
Edge TPUs support 8 bit integer math only. Training is floating point. That's not a small change. https://coral.withgoogle.com/docs/edgetpu/models-intro/#mode...