News for hackers. Stuff that matters.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by rsynnott on Saturday August 31, 2024 @02:08PM (#41410629)
Okay, I mean I feel like they’re not that mysterious. Like, there are probably only five or six candidates.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by nosefurhairdo on Saturday August 31, 2024 @03:52PM (#41411447)
> Although the names of the mystery AI whales are not known, they are likely to include Amazon, Meta, Microsoft, Alphabet, OpenAI, or Tesla.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by terafo on Saturday August 31, 2024 @04:15PM (#41411624)
Why mention Microsoft twice?
Re:Nearly half of Nvidia's revenue comes from four mystery w... by mensetmanusman on Saturday August 31, 2024 @04:20PM (#41411657)
The departments aren’t talking, so they accidentally made two orders.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by fnordpiglet on Saturday August 31, 2024 @04:53PM (#41411871)
I’d note Microsoft needs OpenAI a hell of a lot more than OpenAI needs Microsoft. I’d actually pivot that to be why mention OpenAI twice.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by amluto on Saturday August 31, 2024 @06:20PM (#41412468)
How so? As far as I can tell, Microsoft has a large equity interest in OpenAI, and OpenAI has a lot of cloud credits usable on Microsoft’s cloud. I don’t think those credits are transferable to other providers.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by fnordpiglet on Sunday September 01, 2024 @12:24AM (#41414282)
The value in the proposition is OpenAI IP. Money and data centers are commodities easily replaced, especially when you hold the IP everyone wants a piece of.

The arrangement is mutually beneficial, but the owner of the IP holds the cards.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by FridgeSeal on Saturday August 31, 2024 @06:18PM (#41412460)
OpenAI doesn’t operate without the enormous amounts of funding MS gives it.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by adwi on Saturday August 31, 2024 @06:41PM (#41412593)
I think a lot of institutions and people would love the chance to give them money.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by noirbot on Saturday August 31, 2024 @06:56PM (#41412670)
But how many of them have hot data centers to offer? Google is a direct competitor, so Oracle or Amazon are kinda the only other two big options to offer them what MS is right now.

If MS drops OpenAI, it's not like they can just seamlessly pivot to running their own data centers with no downtime, even with pretty high investment.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by fnordpiglet on Sunday September 01, 2024 @12:53AM (#41414402)
A relationship that’s mutually beneficial needn’t be symmetric. Microsoft’s relationship is fairly commoditized - money and GPUs. OpenAI controls the IP that matters.

I’d note that the supplier of GPUs is Nvidia, who also offers cloud GPU services and doesn’t have a stake in the GCP, Azure, AWS behemoth battle. I’d actually see that as a more natural less middle man relationship.

The real value azure brings is enterprise compliance chops. However IMO aws bedrock seems to be a more successful enterprise integration point. But they’re all commodity products and don’t provide the value OpenAI provides to the relationships.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by 7thpower on Saturday August 31, 2024 @02:38PM (#41410847)
Companies like Coreweave who lease accelerators may make this analysis less straight forward than it appears.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by jeffbee on Saturday August 31, 2024 @03:53PM (#41411465)
Even less straightforward since Nvidia funds Coreweave.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by kergonath on Saturday August 31, 2024 @02:09PM (#41410637)
And regardless of who the 4 are, the two others in that list of six candidates are most likely not too far behind.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by gpm on Saturday August 31, 2024 @02:41PM (#41410863)
The most interesting thing to discover would be if one of them is that far behind, because they're succeeding on their own/someone not-Nvidia's silicon.

The public in-house projects that I'm aware of (but as far as I know haven't fully replaced demand for Nvidia GPUs) include:

- Google's "TPU" (in production, publicly rentable)

- Amazon/AWS's "Trainium" (in production, publicly rentable)

- Meta's "MTIA" (in production)

- Microsoft's "Maia 100" (I'm unclear on their status)

- Tesla's "D1" (I'm unclear on their status)

Re:Nearly half of Nvidia's revenue comes from four mystery w... by solidasparagus on Saturday August 31, 2024 @03:58PM (#41411495)
It's not just the hardware, it's the software stack too and my understanding is that they aren't very good. Even TPUs aren't great if you aren't either (1) doing something extremely standard and a little bit old (e.g. not forefront of research and the stack has already been optimized for your model) or (2) in Google with access to the people who build the stack.

Maybe it is working for Meta or Tesla where things can be vertically integrated, but for the public clouds, they have to buy NVIDIA for their customers.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by rajman187 on Saturday August 31, 2024 @03:50PM (#41411435)
MTIA will be for inference initially. Another to add to the list is wafer maker Cerebras

https://www.forbes.com/sites/craigsmith/2024/08/27/cerebras-...

Re:Nearly half of Nvidia's revenue comes from four mystery w... by jsheard on Saturday August 31, 2024 @03:09PM (#41411102)
Are these distinct architectures, or is it an ARM situation where nearly everyone is gluing the same IP cores together in slightly different configurations?
Re:Nearly half of Nvidia's revenue comes from four mystery w... by zaphar on Saturday August 31, 2024 @03:18PM (#41411184)
I'm not sure about all of them but Google's TPU is custom to them and not shared architecture.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by pclmulqdq on Saturday August 31, 2024 @03:34PM (#41411304)
They are distinct architectures, but mostly do the same thing. Pretty much all of them have a few small control cores that run matrix multiply and vector reduction units. The instruction set on all of them is different, but the broad strokes of the architecture are the same.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by taktoa on Saturday August 31, 2024 @03:17PM (#41411176)
Definitely not an ARM situation.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by HarHarVeryFunny on Saturday August 31, 2024 @03:01PM (#41411027)
xAI 100K H100 cluster would be ~$2B

Is this trailing year NVIDIA sales, or order book ?

Re:Nearly half of Nvidia's revenue comes from four mystery w... by jsnell on Saturday August 31, 2024 @03:16PM (#41411167)
The numbers in the article? They're neither. They're the revenue for just Q2, not the full trailing year.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by bradleyjg on Saturday August 31, 2024 @04:52PM (#41411862)
Does TSM have the capacity to scale up anything new right now?
Re:Nearly half of Nvidia's revenue comes from four mystery w... by ninetyninenine on Saturday August 31, 2024 @02:23PM (#41410739)
Let's name them, and why?
Re:Nearly half of Nvidia's revenue comes from four mystery w... by jmathai on Saturday August 31, 2024 @02:26PM (#41410758)
Anyone building the largest of LLMs including Alphabet, Meta, OpenAI, Anthropic
Re:Nearly half of Nvidia's revenue comes from four mystery w... by qwertox on Saturday August 31, 2024 @02:43PM (#41410883)
I doubt Anthropic builds their own GPU datacenter.

They might buy some, but I think that Google, Meta, Microsoft and Amazon and will be the ones buying in large batches to enable companies like Anthropic (and themselves) to scale up to world wide inferencing demands, as well as generally offering the most efficient GPUs to their customers.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by jmathai on Saturday August 31, 2024 @02:48PM (#41410926)
Think they're renting GPUs from the cloud providers?

Very plausible. I'm not sure at which point it makes economic sense to buy the GPUs and build out the infrastructure to continually be training something like Claude.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by alephnerd on Saturday August 31, 2024 @02:54PM (#41410971)
> Think they're renting GPUs from the cloud providers

It's a major reason why they raised with Amazon [0]

There are actually a LOT of other large companies that participated in Anthropic's round but haven't announced it publicly.

[0] - https://www.aboutamazon.com/news/company-news/amazon-anthrop...

Re:Nearly half of Nvidia's revenue comes from four mystery w... by VirusNewbie on Saturday August 31, 2024 @03:51PM (#41411440)
They train on GCP now.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by bigyikes on Saturday August 31, 2024 @05:22PM (#41412109)
They also raised with Google, interestingly.

https://www.wsj.com/tech/ai/google-commits-2-billion-in-fund...

Re:Nearly half of Nvidia's revenue comes from four mystery w... by alephnerd on Saturday August 31, 2024 @07:23PM (#41412818)
Yep! You're right! I think they were initially using AWS though.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by spwa4 on Saturday August 31, 2024 @02:30PM (#41410791)
I think you should probably add all large cloud providers. Amazon and Microsoft should be in the list.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by azinman2 on Saturday August 31, 2024 @02:38PM (#41410845)
And we know OpenAI uses Microsoft for GPUs. My guess is Anthropic is similarly not owning their own data centers; didn’t they get a bunch of money from google? It’s probably being spent there.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by alephnerd on Saturday August 31, 2024 @03:00PM (#41411019)
They use AWS.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by VirusNewbie on Saturday August 31, 2024 @03:53PM (#41411463)
No they don’t.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by azinman2 on Saturday August 31, 2024 @10:24PM (#41413710)
Care to substantiate?

According to Wikipedia:

In September 2023, Amazon announced an investment of up to $4 billion, followed by a $2 billion commitment from Google in the following month.

They probably use both. Almost certainly most of this is credits in their cloud platforms.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by danjl on Saturday August 31, 2024 @03:40PM (#41411357)
Oh, I would not forget governments. The NSA basically paid for Kepler with a single purchase...
Re:Nearly half of Nvidia's revenue comes from four mystery w... by marcosdumay on Saturday August 31, 2024 @02:31PM (#41410798)
I believe the GP refers to Google, Facebook, Amazon and Microsoft.

And while yeah, that's probably them, I'd put a non-trivial chance of some government intelligence organization to make it into the top 3.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by miki123211 on Saturday August 31, 2024 @02:44PM (#41410889)
ANd possibly some less-known companies, as fronts for the Chinese.
Re:Nearly half of Nvidia's revenue comes from four mystery w... by alephnerd on Saturday August 31, 2024 @03:02PM (#41411043)
> less-known companies, as fronts for the Chinese.

Not at that size. That is VERY on the nose sanctions evasion.

Such sanctions evasions tend to use multiple smaller parties doing purchases and then reselling.

Re:Nearly half of Nvidia's revenue comes from four mystery w... by alephnerd on Saturday August 31, 2024 @03:00PM (#41411023)
> I'd put a non-trivial chance of some government intelligence organization to make it into the top 3

Most likely DoE. TLAs purchase indirectly (or use other federal agencies in the DoD as a front)

Re:Nearly half of Nvidia's revenue comes from four mystery w... by ghshephard on Saturday August 31, 2024 @02:40PM (#41410859)
I’m guessing a massive amount is for inference for whatsapp, and the original goal was making for relevant instagram - and of course the massive Llama model training - my guess is Facebook is a relatively small component of Meta’s overall use of GPUs. Feed recommendations? (unless you were using facebook as a holder for Meta?)
Re:Nearly half of Nvidia's revenue comes from four mystery w... by vineyardmike on Saturday August 31, 2024 @03:31PM (#41411277)
It’s absolutely not WhatsApp. It’s their recommendation engines. They’ve publicly stated they’re buying enough GPUs to have the spare capacity to train another “reels” sized product for when the opportunity emerges.

(They absolutely use it as a holder for Meta)