News for hackers. Stuff that matters.
Re:Every Model Learned by Gradient Descent Is Approximately ... by nuclearnice1 on Sunday December 06, 2020 @01:46PM (#25325565)
Great explanation of the intuitive understanding of the path kernel which seems to be the main takeaway from this paper.

One minor technical correction, the proof relief on the continuous model gradient flow not SGD. So it’s proven for GD and likely true for SGD and your intuitive explanation likely still holds, but it’s not obvious.

Re:Every Model Learned by Gradient Descent Is Approximately ... by cs702 on Sunday December 06, 2020 @03:02PM (#25326189)
You're absolutely right. I substituted SGD for GD without giving it any thought because everyone uses SGD!
Re:Every Model Learned by Gradient Descent Is Approximately ... by QuesnayJr on Sunday December 06, 2020 @03:31PM (#25326368)
They sketch the argument for SGD, but they don't know if it actually holds (see Remark 5 in the paper).