Re:Google breaks the trillion-parameter ceiling with the Swi...
by igorkraw on Monday January 18, 2021 @01:54PM
(#25825137)
It basically means you need to learn about double descent:
https://twitter.com/hippopedoid/status/1243229024085835779
and regularize your models well, implicitly or explicitly. Your model might otherwise memorize the data instead of learning features.
What does that mean?