News for hackers. Stuff that matters.

A comment on Google supercharges machine learning tasks with TPU custom chip. Parent comment

Re:Google supercharges machine learning tasks with TPU custo... by euyyn on Wednesday May 18, 2016 @04:45PM (#11725787)
Do you reckon that means it's using small floats?
[ Parent | on HN ]
Re:Google supercharges machine learning tasks with TPU custo... by protomok on Wednesday May 18, 2016 @05:00PM (#11725925)
Based on recent blog posts from some Google folks regarding quantizing neural nets I'm going to guess 8-bit fixed point.

For example -> https://petewarden.com/2016/05/03/how-to-quantize-neural-net...

[ Parent | on HN ]
Re:Google supercharges machine learning tasks with TPU custo... by nhaehnle on Wednesday May 18, 2016 @06:59PM (#11726696)
Probably.

In addition, there's a lot of literature on optimizing hardware implementations of fundamental arithmetic operations like addition and multiplication. I recall seeing a paper a while ago which talked about reducing the number of gates by allowing some bounded imprecision in the results - unfortunately, I don't remember the title right now, but it sounds like that's what they may be doing.

[ Parent | on HN ]
Re:Google supercharges machine learning tasks with TPU custo... by tcarey83 on Wednesday May 18, 2016 @11:10PM (#11727697)
When I first read the blog post earlier in the day, it actually said they were using 8-bits. I remember it because it seem quite small to me.
[ Parent | on HN ]
Re:Google supercharges machine learning tasks with TPU custo... by pygy_ on Wednesday May 18, 2016 @05:01PM (#11725929)
Or small ints, possibly working in log space.
[ Parent | on HN ]