Re:RedPajama: Reproduction of Llama with Friendly License (S...
by ipsum2 on Monday April 17, 2023 @04:09PM
(#35605908)
Huh? There's plenty of open source LLMs. Pythia, GPT-NeoX, GPT-J, GPT-2, BLOOM-176, are ones I can think of off the top of my head. Pythia is the best performing one IIRC.
[ Parent
| on HN ]
Re:RedPajama: Reproduction of Llama with Friendly License (S...
by bradknowles on Monday April 17, 2023 @08:32PM
(#35608517)
Hmm. Would you be able to combine these LLMs? Or are they already supersets of each other?
Re:RedPajama: Reproduction of Llama with Friendly License (S...
by MacsHeadroom on Monday April 17, 2023 @09:37PM
(#35608933)
Pythia was trained on only 300B tokens and is pretty dumb compared to LLaMA.
Pythia 13B is worse than LLaMA-7B and requires double the resources.
Re:RedPajama: Reproduction of Llama with Friendly License (S...
by Tepix on Tuesday April 18, 2023 @11:12AM
(#35615059)
Not all use cases need GPT-4 level performance. I'd argue that even LLaMA-7B is quite limited. Also, new and improved models are being released all the time.
Re:RedPajama: Reproduction of Llama with Friendly License (S...
by lhl on Tuesday April 18, 2023 @09:08AM
(#35613494)
I started keeping a list btw, there are about 20 completely open (Apache 2.0, BSD, MIT, CC-BY) 1B+ parameter foundational LLMs at the moment: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp...