Re:Every Model Learned by Gradient Descent Is Approximately ...
by nuclearnice1 on Sunday December 06, 2020 @01:46PM
(#25325565)
Great explanation of the intuitive understanding of the path kernel which seems to be the main takeaway from this paper.
One minor technical correction, the proof relief on the continuous model gradient flow not SGD. So it’s proven for GD and likely true for SGD and your intuitive explanation likely still holds, but it’s not obvious.
One minor technical correction, the proof relief on the continuous model gradient flow not SGD. So it’s proven for GD and likely true for SGD and your intuitive explanation likely still holds, but it’s not obvious.