Re:OpenAI is giving Microsoft exclusive access to its GPT-3 ...
by sinenomine on Saturday October 10, 2020 @03:47PM
(#24741363)
I think your first sentence is an answer to every thread of this nature. The rarity of this sentiment and people's sentiment towards it are self-evident.
Training a language model similar to GPT-3 in capability, but cheaper (and smaller parameter-count-wise) is entirely possible for a committed team, if one applies more modern architecture and finds cheaper training infrastructure. For example a 1080ti cluster or a few graphcore servers. Also one does not have to start from scratch, the initial model can be distilled from a publicly available snapshot of one of the larger language models.
In fact, as we are talking, several teams are racing towards this goal. I see this as the main reason for prevalence of non-constructive comments in such threads - everybody with constructive attitude has found a team or founded their own.
If you are interested in these questions, get in touch.
Training a language model similar to GPT-3 in capability, but cheaper (and smaller parameter-count-wise) is entirely possible for a committed team, if one applies more modern architecture and finds cheaper training infrastructure. For example a 1080ti cluster or a few graphcore servers. Also one does not have to start from scratch, the initial model can be distilled from a publicly available snapshot of one of the larger language models.
In fact, as we are talking, several teams are racing towards this goal. I see this as the main reason for prevalence of non-constructive comments in such threads - everybody with constructive attitude has found a team or founded their own.
If you are interested in these questions, get in touch.