News for hackers. Stuff that matters.
Re:Tiny Language Models Come of Age by whimsicalism on Sunday October 08, 2023 @04:49PM (#37814275)
Saying that information density will increase is very different from saying model sizes will decrease of a similar order of magnitude.

Regardless, this isn't the trend we've seen with software RAM usage - information density is only decreasing as hardware becomes more abundant and comparatively less engineering effort goes into fitting everything into as small a portion as possible.

There are no comments on this comment yet.