News for hackers. Stuff that matters.
Re:Giving GPT-3 a Turing Test by rainbow-road on Sunday July 19, 2020 @08:42AM (#23888951)
I see this "175B parameters" figure tossed around a lot: what does that mean? Does the model literally consist of 175 billion 32-bit floats?

(Sorry for the very basic question.)

Re:Giving GPT-3 a Turing Test by qayxc on Sunday July 19, 2020 @09:14AM (#23889118)
You would be correct.

It's ~650GiB of data. The entire English version of Wikipedia is about 20GiB and the Encyclopædia Britannica is only about 300MiB (measured in ASCII characters) [1].

[1] https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons

Re:Giving GPT-3 a Turing Test by pradn on Sunday July 19, 2020 @12:05PM (#23890246)
The English Wikipedia text (no images) is about ~20 GB compressed, but ~60 GB uncompressed.
Re:Giving GPT-3 a Turing Test by qayxc on Sunday July 19, 2020 @01:25PM (#23890828)
I only use the sources available to me at the time.

Another value published by Wikipedia is 30GiB [1] as of 2020, which includes punctuation and markup.

I explicitly put the measurement unit as ASCII characters. If you have a better source for your size (remember: ASCII characters for article words only, no markup), feel free to post it.

[1] https://en.wikipedia.org/wiki/Wikipedia:Size_in_volumes