Re:Trade-Offs in Automatic Differentiation: TensorFlow, PyTo...
by ogogmad on Saturday December 25, 2021 @03:42PM
(#29686780)
The subgradient in convex analysis is a special case of the Clarke gradient. The subgradient is precisely the Clarke gradient for convex functions. Convex functions are always locally Lipschitz except in weird cases.
[edit]
Question: Are there numerical applications in which the subgradient is actually computed, or is it a purely analytical tool?
Re:Trade-Offs in Automatic Differentiation: TensorFlow, PyTo...
by agnosticmantis on Saturday December 25, 2021 @05:12PM
(#29687459)
(Stochastic) subgradient methods are used in practice to optimize non-differentiable convex functions. They have a slower convergence rate than (stochastic) gradient descent though.
See for example : https://www.stat.cmu.edu/~ryantibs/convexopt-F15/lectures/07...
Re:Trade-Offs in Automatic Differentiation: TensorFlow, PyTo...
by fault1 on Saturday December 25, 2021 @05:51PM
(#29687774)
yes, see:
https://juliadiff.org/ChainRulesCore.jl/dev/maths/nondiff_po...
and related neat usages in set based optimization methods in MathOptInterface (part of JuMP.jl): https://matbesancon.xyz/post/2020-12-24-chains_sets2/
[0] https://en.m.wikipedia.org/wiki/Subderivative