News for hackers. Stuff that matters.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by kevmo314 on Saturday July 18, 2020 @11:48PM (#23886838)
> That post was generated using GPT-3 and 175 billion parameters of pre-existing human writing, contextualized, distilled, and cross-referenced with terminology we've agreed on for centuries.

So basically like DNA?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by saberience on Sunday July 19, 2020 @09:13AM (#23889110)
DNA is nothing like the training of GPT. DNA does not encord a massive amount of statistics of words and language and how concepts, words, etc, relate to one another.

All DNA does it encode for how to grow, build, and mantain a human body. That human body has the potential to learn a language and communicate, but if you put a baby human inside an empty room and drop in food, it will never learn language and never communicate. DNA isn't magic and comparing "millions of years of evolution" of DNA is nothing like the Petabytes of data that GPT-3 needs to operate.

Again DNA has no knowledge embedded in it, it has no words or data embedded. Data in the sense that we imagine Wikipedia stored in JSON files on a hard disk. DNA stores an algorithm for growth of a human, that's it.

The GPT-3 model is probably > 700GB in size. That is, for GPT to be able to generate text it needs an absolutely massive "memory" of existing text which it can recite verbatim. In contrast, young human children can generate more novel insights with many orders of magnitude less data in "memory" and less training time.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by varjag on Sunday July 19, 2020 @12:24PM (#23890377)
Since literacy or human knowledge isn't encoded in DNA, it's nothing like it.