News for hackers. Stuff that matters.
Re:Outrageously Large Neural Networks: The Sparsely-Gated Mi... by yorwba on Monday January 30, 2017 @08:11AM (#13520806)
Couln't you use an initialization pattern that includes all the weights of the smaller layer in the larger layer? This would keep the behavior of a subset of units exactly the same, at least at initialization time.

There are no comments on this comment yet.