News for hackers. Stuff that matters.
Re:Double Descent in Human Learning by johndough on Wednesday April 26, 2023 @09:02AM (#35712991)
Has anyone been able to reproduce double descent without trying to invert a seriously ill-posed linear system? Showing that a solver has numerical issues which go away when enough parameters are introduced does not really prove anything. This looks much more like regularization though floating point rounding.

By the way, the linked Colab notebook is missing a "self." in front of the "lamb" in the unused regularization branch of the fit function.

Re:Double Descent in Human Learning by sebzim4500 on Wednesday April 26, 2023 @09:30AM (#35713361)
OpenAI paper on double descent: https://arxiv.org/pdf/1912.02292.pdf

They show it happens across a variety of architectures and real world datasets