Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on The Secret Sauce behind 100K context window in LLMs: all tricks in one place. Parent comment
OpenAI didn’t build a bespoke supercomputer, but trained on Azure (with a preferential contract thanks to their investors): https://openai.com/gpt-4
> GPT-4 was trained on Microsoft Azure AI supercomputers
Cohere trained on GCP: https://techcrunch.com/2021/11/17/google-cloud-teams-up-with...
> Aidan Gomez, co-founder and CEO at Cohere[:] “We scrape the data to train these big models, we train them on massive TPU pods”
Stability AI trains on AWS: https://aws.amazon.com/blogs/machine-learning/stability-ai-b...
> With Amazon SageMaker, Stability AI will build AI models on compute clusters with thousands of GPU or AWS Trainium chips
Given Anthropic’s recent partnership announcement, they likely train on GCP nowadays: https://www.anthropic.com/index/anthropic-partners-with-goog...
> Anthropic will leverage Google Cloud's cutting-edge GPU and TPU clusters to train, scale, and deploy its AI systems
There are no comments on this comment yet.
OpenAI didn’t build a bespoke supercomputer, but trained on Azure (with a preferential contract thanks to their investors): https://openai.com/gpt-4
> GPT-4 was trained on Microsoft Azure AI supercomputers
Cohere trained on GCP: https://techcrunch.com/2021/11/17/google-cloud-teams-up-with...
> Aidan Gomez, co-founder and CEO at Cohere[:] “We scrape the data to train these big models, we train them on massive TPU pods”
Stability AI trains on AWS: https://aws.amazon.com/blogs/machine-learning/stability-ai-b...
> With Amazon SageMaker, Stability AI will build AI models on compute clusters with thousands of GPU or AWS Trainium chips
Given Anthropic’s recent partnership announcement, they likely train on GCP nowadays: https://www.anthropic.com/index/anthropic-partners-with-goog...
> Anthropic will leverage Google Cloud's cutting-edge GPU and TPU clusters to train, scale, and deploy its AI systems