Pruning AI networks without impacting performance

Re:Pruning AI networks without impacting performance by vanderZwan on Wednesday December 13, 2017 @05:50AM (#15913436)

https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf

Looks pretty close, yeah.

So this is basically dividing the network into two groups, and give each group a second weight: one for those that are on, zero for those that are off.

Has anyone tried more variants of this? Using random bundling + various levels of amplification of such bundles, so that backpropagation affects various networks differently?

[ Parent | on HN ]

Re:Pruning AI networks without impacting performance by visarga on Friday December 15, 2017 @03:31PM (#15934822)

There is also this: Mixture of experts. A massive neural net from Google, with multiple submodules that are activated depending on input.

https://medium.com/@thoszymkowiak/google-brains-new-super-fa...

[ Parent | on HN ]