Re:All-optical machine learning using diffractive deep neura...
by obastani on Monday August 06, 2018 @01:59PM
(#17699961)
Though the other comments are correct, I want to point out that you can get some nontrivial behavior with only linear functions. For example, low-rank matrix factorization is kind of like a neural network
f(x) = U * V * x,
where U is an n by k matrix and V is a k by m matrix, where k is much smaller than n and m. Basically, we are constraining the set of allowed linear transformations, which is a form of regularization. Convolutional layers in neural networks similarly restrict the allowed linear transformations.
Nevertheless, the power of linear neural networks is far less than that off nonlinear networks.
f(x) = U * V * x,
where U is an n by k matrix and V is a k by m matrix, where k is much smaller than n and m. Basically, we are constraining the set of allowed linear transformations, which is a form of regularization. Convolutional layers in neural networks similarly restrict the allowed linear transformations.
Nevertheless, the power of linear neural networks is far less than that off nonlinear networks.