Re:Gated Linear Networks
by xiaodai on Monday June 15, 2020 @10:17PM
(#23535339)
what stops you from using a few convolution layers and then use the GLN in the last layer? You achieve the best of both worlds if your theory is true.
[ Parent
| on HN ]
Re:Gated Linear Networks
by fxtentacle on Tuesday June 16, 2020 @03:04AM
(#23536622)
That would work, but it would likely re-introduce the catastrophic forgetting problem, because now the GLN is dependent on an intermediate representation determined by conv layers.