News for hackers. Stuff that matters.
Re:Google “We Have No Moat, and Neither Does OpenAI” by lapinot on Thursday May 04, 2023 @11:02AM (#35816464)
> Seems like wishful thinking to think decentralized open source can beat centralized models that cost 100M+ to train!

Because surely price = quality. Solid argumentation there.

Re:Google “We Have No Moat, and Neither Does OpenAI” by Mike_12345 on Friday May 05, 2023 @04:40AM (#35826736)
Yes, price = quality because they require so much computing resources to train. GPT-3 required the equivalent of one month of training on ~1000 A100 GPUs. That's $10 million for the GPU hardware alone, not including power ($10k each)
Re:Google “We Have No Moat, and Neither Does OpenAI” by sashank_1509 on Saturday May 06, 2023 @11:40AM (#35842416)
You’re right but I’d just like to add, GPT-3 probably required 1000’s of GPUs. OpenAI is known to have the largest cluster 16k+ A100 GPUs and most of them were used for the major model training.