Re:Matrix Calculus for Deep Learning
by psv1 on Friday November 29, 2019 @02:39AM
(#21662776)
Just want to point out that you're asking about "ML" but your questions are about neural networks/deep learning which is only a subset of machine learning.
> is there any math behind ML at all?
Yes. There is a lot of research in this area (some people argue it's excessive at the moment). You have the correct intuition that the answers aren't black and white all the time. For example, there are solid reasons to choose relu activations over tanh. Or to build certain types of network architectures for certain tasks. That doesn't mean that you can immediately calculate what would happen if you switch from one activation to another without running your network.
> is there any math behind ML at all?
Yes. There is a lot of research in this area (some people argue it's excessive at the moment). You have the correct intuition that the answers aren't black and white all the time. For example, there are solid reasons to choose relu activations over tanh. Or to build certain types of network architectures for certain tasks. That doesn't mean that you can immediately calculate what would happen if you switch from one activation to another without running your network.