News for hackers. Stuff that matters.
Re:Light computes any desired linear transform without a dig... by WORLD_ENDS_SOON on Monday September 27, 2021 @10:20AM (#28670947)
Deep learning combines linear functions together, but the way in which they are combined is always nonlinear. If the way in which they were combined were linear, the resulting function would also be linear (a linear combination of linear functions is linear), and you'd have linear regression. Specifically the final layer of a deep neural network is often linear but the nodes in middle layers are often Rectified Linear Units which have a nonlinear "max" function applied to a linear function. Without these "max" functions you'd be able to flatten all the layers into one. Other common nonlinear functions you'll find in a deep neural net are sigmoid functions like tanh. These functions are sometimes called "activation functions".
Re:Light computes any desired linear transform without a dig... by Buttons840 on Monday September 27, 2021 @04:15PM (#28675457)
Yes, there is a non-linear function between each linear function. The alternating back and forth between linear and non-linear is important.