Not that this is relevant to OP, but there are also non-differentiable descent (edit: NOT descent, but rather just general optimization) methods; https://optimization.mccormick.northwestern.edu/index.php/No...