Re:Ask HN: Why do Sam and Greg need Microsoft?
by hypoxia87 on Tuesday November 21, 2023 @10:46AM
(#38365222)
I think it comes down to prioritizing speed of execution in the race for AGI, especially in the context of two constrained resources:
1. Compute (H100s are almost impossible to get ahold of, especially in the quantities they'd need to train a competitive model)
2. Data (OpenAI has a very rich set of user data from ChatGPT that is necessary for creating a model like GPT-4; a new venture would lack this data -- the cold start problem -- but Microsoft may* have similarly usable data from non-enterprise Bing Chat)
* Someone please feel free to prove me wrong on this - I'm not aware of a MS policy saying they won't use Bing Chat prompts for training, but it could very well exist.
1. Compute (H100s are almost impossible to get ahold of, especially in the quantities they'd need to train a competitive model)
2. Data (OpenAI has a very rich set of user data from ChatGPT that is necessary for creating a model like GPT-4; a new venture would lack this data -- the cold start problem -- but Microsoft may* have similarly usable data from non-enterprise Bing Chat)
* Someone please feel free to prove me wrong on this - I'm not aware of a MS policy saying they won't use Bing Chat prompts for training, but it could very well exist.