Re:Emergent Chip Vastly Accelerates Deep Neural Networks
by nl on Wednesday January 13, 2016 @12:16AM
(#10892442)
[1] is a Google-authored demo running Inception-v3[2] (about twice as accurate as VGG) on Android phones.
I don't know how many parameters Inception v3 has, but I know Google considers it more efficient than VGG ("Although our network is 42 layers deep, our computation cost is only about 2.5 higher than that of GoogLeNet and it is still much more efficient than VGGNet".)
Yes, being able to run big networks is great. But ultimately it's what you do with it, and a 3% error rate on ImageNet is a pretty compelling argument that size isn't the only factor.
[1] https://github.com/tensorflow/tensorflow/tree/master/tensorf...
Of course it can. The problem is with the size of the network you might want to run.
From your first link, which is about single character level image processing
> We needed to develop a very small neural net, and put severe limits on how much we tried to teach it—in essence, put an upper bound on the density of information it handles.
And from the voice training paper:
> While our server-based model has 50M parameters (k = 4, nh = 2560, ni = 26 and no = 7969), to reduce the memory and computation requirement for the embedded model, we experimented with a variety of sizes and chose k = 6, nh = 512, ni = 16 and no = 2000, or 2.7M parameters
AlexNet is, what, 60M+? VGG is pretty big too.
I guess I wasn't too clear. Yes, there are good results from nets that we can run on mobile devices in realtime. We do, however, want to run significantly larger nets.