Same thing. You need everything to be differentiable in order to run gradient descent. To first approximation, training a neural network is just gradient descent.