News for hackers. Stuff that matters.
Re:Nvidia H100 GPU Shipments by Customer by mtmail on Saturday December 02, 2023 @06:59PM (#38503463)
"Nvidia spends around $3,320 to manufacture a single H100 unit, which is then sold to end customers for a price ranging from $25,000 to $30,000." https://www.techspot.com/news/99839-nvidia-garners-reported-...

I'm sure a customer ordering 150.000 will get a discount, but how much? (150.000 x 25.000 = 3.75 billion)

Re:Nvidia H100 GPU Shipments by Customer by marcus0x62 on Saturday December 02, 2023 @08:31PM (#38503965)
Also consider that the $3,320 figure appears to be an estimate of the cost to manufacture an H100, which doesn’t encompass all of Nvidia’s COGS (R&D, warranty reserve, etc.)
Re:Nvidia H100 GPU Shipments by Customer by freedomben on Saturday December 02, 2023 @08:56PM (#38504091)
agreed there probably is a bulk discount of some sort, but Nvidia has them by the balls because there aren't really other options, so they pressure to give good bulk discounts would be lessened a bit
Re:Nvidia H100 GPU Shipments by Customer by dotnet00 on Saturday December 02, 2023 @09:18PM (#38504185)
Not only are there no other options, but Nvidia can't make them fast enough, so even more reason to not really bother with significant bulk discounts.
Re:Nvidia H100 GPU Shipments by Customer by samspenc on Saturday December 02, 2023 @09:12PM (#38504162)
This makes sense, let's say that R&D per unit is a similar amount, that would totally explain how Nvidia is able to generate insane 50%+ net profit margins (74% gross margin), higher than FAANG and unheard of for hardware companies [1][2]

  [1] https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-third-quarter-fiscal-2024
  [2] https://www.reddit.com/r/Infographics/comments/1825y3f/nvidias_q3_earnings_visualized/
Re:Nvidia H100 GPU Shipments by Customer by fulafel on Sunday December 03, 2023 @02:41AM (#38505543)
It's seems fourth (at best) hand info, the source being a tweet by an analyst referencing another analyst.

NV does not manufacture chips, wonder if charitably they meant that they pay this much to the manufacturer? In that case this would be their price, I guess in a way "manufacturing cost" but quite misleading since the chip manufacturer probably has big margins.

Re:Nvidia H100 GPU Shipments by Customer by sebmellen on Saturday December 02, 2023 @06:22PM (#38503242)
Incredible that Meta has 1.5x the amount as Google and Amazon combined. For all the criticisms of Zuckerberg, Meta is an insanely strong and well-aligned organization, and their open-source contributions are unparalleled.
Re:Nvidia H100 GPU Shipments by Customer by thelastgallon on Saturday December 02, 2023 @07:59PM (#38503784)
Google has TPU, might not need as many as others.
Re:Nvidia H100 GPU Shipments by Customer by valianteffort on Saturday December 02, 2023 @10:41PM (#38504601)
Likewise Tesla are at the bottom of the list but already have a significant investment towards their in-house silicon.

Buying H100's is almost necessary if only to assure you're not compute starved.

Re:Nvidia H100 GPU Shipments by Customer by 1letterunixname on Sunday December 03, 2023 @11:44PM (#38513819)
Motion is not necessarily progress. (Picture of a rocking horse.)

FOSS is all well-and-good, but that doesn't directly translate into profits.

Except it's based on a vague, "build it and they will spend" wing and a prayer cargo cult rather than fulfilling any specific, pressing need.

Morale is low as there are round after round of layoffs.

Re:Nvidia H100 GPU Shipments by Customer by bradknowles on Wednesday December 06, 2023 @10:35PM (#38552540)
Amazon has their own in-house architectures that are designed for high performance AI training and execution.

So maybe that's why they don't have so much in the way of stuff from NVIDIA?

Re:Nvidia H100 GPU Shipments by Customer by hooloovoo_zoo on Saturday December 02, 2023 @08:58PM (#38504097)
How would this 'Omdia' know who got how many GPUs?
Re:Nvidia H100 GPU Shipments by Customer by jiggawatts on Saturday December 02, 2023 @08:58PM (#38504098)
There is no source on this info, which would be commercial in confidence at the very least.

Also: how come Chinese companies are on this list!? I thought H100 exports to China were banned by the Biden administration!

Re:Nvidia H100 GPU Shipments by Customer by justinclift on Sunday December 03, 2023 @02:18AM (#38505466)
Possibly things that were shipped before the cut-off date?
Re:Nvidia H100 GPU Shipments by Customer by gymbeaux on Sunday December 03, 2023 @06:06PM (#38511714)
Ban took effect a couple weeks ago
Re:Nvidia H100 GPU Shipments by Customer by Lammy on Saturday December 02, 2023 @08:59PM (#38504103)
*Who did NVIDIA sell its H100 to that isn't classified?
Re:Nvidia H100 GPU Shipments by Customer by hepinhei on Sunday December 03, 2023 @04:14AM (#38505862)
I see many Chinese companies there. Wasn't nvidia not allow to sell this kind of gpus to China?
Re:Nvidia H100 GPU Shipments by Customer by gymbeaux on Sunday December 03, 2023 @06:00PM (#38511668)
That ban only took effect around Thanksgiving (late November 2023).
Re:Nvidia H100 GPU Shipments by Customer by MichaelRazum on Sunday December 03, 2023 @04:37AM (#38505932)
As I understand a H100 (SXM) gets you about 4 Petaflops on FP8. So 150k H100 is about 600 Exaflops on FP8. That is huge. Maybe enough to get AGI?
Re:Nvidia H100 GPU Shipments by Customer by jimsimmons on Sunday December 03, 2023 @04:47AM (#38505962)
A can of coke is a few inches. 150K cans of coke is huge. That's enough to get to the moon?
Re:Nvidia H100 GPU Shipments by Customer by artisin on Sunday December 03, 2023 @06:42AM (#38506350)
Well, if I could bottle up my feelings about AI hype, they'd fizz over like a can of coke.
Re:Nvidia H100 GPU Shipments by Customer by MichaelRazum on Sunday December 03, 2023 @10:02AM (#38507525)
Maybe, if you already reached the atmosphere with it. Obviously, current AI tech scales with compute. It "may" be the case, that just more compute is enough to reach the moon..
Re:Nvidia H100 GPU Shipments by Customer by gymbeaux on Sunday December 03, 2023 @06:05PM (#38511712)
It’ll be interesting to see how long these GPUs are in use, AND what the secondary market will look like in 5 years. Will NVidia shaft Microsoft and Meta like they shaft consumers re: driver support and other artificial limitations/planned obsolescence.

Theoretically Microsoft for example just needs to train one or two models and then retrain every so often, so I’m skeptical that NVidia will continue to knock it out of the park from a sales perspective. I wouldn’t be caught dead holding NVDA stock right now.

Re:Nvidia H100 GPU Shipments by Customer by 1letterunixname on Sunday December 03, 2023 @11:39PM (#38513800)
Meta committed to spend $20 B per year on servers for 2023 and 2024. A large fraction of that is going to Nvidia for AI-powered, OCP-derived custom boxes that accept H100's.

The hilarious part is Zuck can't articulate a strategy for this spending at all.