Re:Outrageously Large Neural Networks: The Sparsely-Gated Mi...
by yorwba on Monday January 30, 2017 @08:11AM
(#13520806)
Couln't you use an initialization pattern that includes all the weights of the smaller layer in the larger layer? This would keep the behavior of a subset of units exactly the same, at least at initialization time.
[ Parent
| on HN ]
There are no comments on this comment yet.