Tiny Language Models Come of Age

Re:Tiny Language Models Come of Age by logicchains on Sunday October 08, 2023 @11:53AM (#37811575)

There are hard limits to what a "brain" of a given size can do. Expecting ensembles of 7B models to match what properly trained 180B model can do at specialised tasks is like expecting a thousand monkeys trained on a specialised task to be able to match human performance.

[ Parent | on HN ]

Re:Tiny Language Models Come of Age by samatman on Sunday October 08, 2023 @12:02PM (#37811653)

If this were true, then GPT4 would be better at chess than Stockfish.

Since it's not, it isn't.

[ Parent | on HN ]

Re:Tiny Language Models Come of Age by Terr_ on Sunday October 08, 2023 @01:04PM (#37812289)

What if there was a large inefficient model just good enough that it could generate small specialized ones and orchestrate them?

Now that I've said it, I wonder if there's any evidence human brains work that way.

[ Parent | on HN ]

Re:Tiny Language Models Come of Age by taneq on Sunday October 08, 2023 @06:13PM (#37814879)

By that logic, humans should be better at chess than Stockfish, but they aren’t.

[ Parent | on HN ]