News for hackers. Stuff that matters.
Re:Google breaks the trillion-parameter ceiling with the Swi... by mensetmanusman on Monday January 18, 2021 @01:45PM (#25825027)
It’s an interesting thought experiment to consider models that have more parameters than there are data points being analyzed.

What does that mean?

Re:Google breaks the trillion-parameter ceiling with the Swi... by igorkraw on Monday January 18, 2021 @01:54PM (#25825137)
It basically means you need to learn about double descent:

https://twitter.com/hippopedoid/status/1243229024085835779

and regularize your models well, implicitly or explicitly. Your model might otherwise memorize the data instead of learning features.

Re:Google breaks the trillion-parameter ceiling with the Swi... by rdlecler1 on Monday January 18, 2021 @01:52PM (#25825116)
All networks are like this. In a network of N nodes you have N^2 potential relationships. That’s just simple paired relationships. You can still go to higher order groups.