Google and Meta have more money and more compute than OpenAI. They have more data, better infrastructure, and more AI researchers. And they have been trying hard to catch up - so far unsuccessfully.
But they have the key problem of all established companies: they built the infrastructure for what is now the wrong thing.
This means to do the right thing at Google would require fighting battles on all sorts of fronts as established mini empires within attempt to latch on to this new perceived opportunity to expand.
to do the right thing at Google would require fighting battles
Determining what the right thing is the hard part. Unfortunately that’s also the case right now at OpenAI.
Re:Greg (2017)
by loveparade on Sunday November 19, 2023 @07:37AM
(#38332108)
Google and Meta have less compute infrastructure purpose-built for training and testing LLMs than OpenAI does. It's something that OpenAI has been investing in for several years, while Google/Meta/etc have just started playing catch up very recently.
Researchers like Ilya were essential several years ago when OpenAI got started, because at that time nobody even knew what to build or how to go about it.
Nah Sam will just use GPT4 to code his vision. That's how it works now, we've been told.
Above all else, what you need to build competitive models is a huge amount of money and access to compute. Money alone doesn't solve that either due to the global hardware shortage. You are competing for a globally limited pool of hardware with all other companies.