Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin
by threeseed on Saturday July 18, 2020 @09:17PM
(#23886217)
> no-one can train GPT-3 in any acceptable time except for those with access to large GPU/ASIC compute power
That's not true.
a) Premium AWS customers, for example, can request to have instance limits removed which then gives you access to all of the AWS GPU-enabled instances available worldwide.
b) People ramble on about GPUs but Intel DLBoost/AVX-512 enabled CPUs can get you comparable performance to a medium end GPU in many situations. That then opens the door to training across all of the cloud and VPS providers.
Money is the limiting factor here not available compute resources.
That's not true.
a) Premium AWS customers, for example, can request to have instance limits removed which then gives you access to all of the AWS GPU-enabled instances available worldwide.
b) People ramble on about GPUs but Intel DLBoost/AVX-512 enabled CPUs can get you comparable performance to a medium end GPU in many situations. That then opens the door to training across all of the cloud and VPS providers.
Money is the limiting factor here not available compute resources.