News for hackers. Stuff that matters.

A comment on NanoGPT. Parent comment

Re:NanoGPT by WithinReason on Wednesday January 11, 2023 @08:08AM (#34337953)
there are no other obvious mechanisms to explain why a smaller model with same or similar architecture would be better than a larger one.

Overfitting?

[ Parent | on HN ]
Re:NanoGPT by Der_Einzige on Wednesday January 11, 2023 @08:14AM (#34338012)
The consensus seems to be that the majority of LMs are undertrained not overfitting though.
[ Parent | on HN ]