Re:Meta is inviting researchers to pick apart the flaws in i...
by plegresl on Monday June 27, 2022 @01:54PM
(#31898153)
The 176B parameter BLOOM model should be available soon: https://bigscience.notion.site/BLOOM-BigScience-176B-Model-a...
Re:Meta is inviting researchers to pick apart the flaws in i...
by option on Monday June 27, 2022 @02:16PM
(#31898458)
yes, looking forward to it especially because it is going to be multilingual by design
Re:Meta is inviting researchers to pick apart the flaws in i...
by timmg on Monday June 27, 2022 @02:22PM
(#31898547)
Dumb question: does 175B parameters mean the number of bytes (or floats?) in the model? Does that also mean you need the whole model in memory to do inference (in practice)?
If so, not many machines have that much RAM. Makes it hard to "play" with.
Re:Meta is inviting researchers to pick apart the flaws in i...
by lostmsu on Monday June 27, 2022 @02:44PM
(#31898861)
float16s or bfloat16s so 2x of that for storage
You can infer using DeepSpeed.
Re:Meta is inviting researchers to pick apart the flaws in i...
by charcircuit on Monday June 27, 2022 @01:56PM
(#31898181)
>However, they did not publish any details on their training data.
Yes they did. It's in the README.
Re:Meta is inviting researchers to pick apart the flaws in i...
by option on Monday June 27, 2022 @02:19PM
(#31898499)
All I can see is “ 1.7 TB of online texts, books, and countless other sources in both English and Russian.”
if there are more details, can you please share a link?
I am worried that “other sources” may contain Yandex.news which is a cesspool of anti-West and anti-Ukraine propaganda
Currently, the largest LLM that is both free and commercially usable (Apache 2.0) is 100B YaLM from Yandex (russian’s copy of Google). However, they did not publish any details on their training data.