News for hackers. Stuff that matters.

A comment on Nvidia Hopper GPU Architecture and H100 Accelerator.

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by algo_trader on Tuesday March 22, 2022 @01:25PM (#30768906)
From the nvidia page,

> 80 billion transistors

> Hopper H100 .. generational leap

> 9x at-scale training performance over A100

> 30x LLM inference throughput

> Transformer Engine .. speed .. 6x without losing accuracy

So another monster chip - same size of the Apple M1-max thingy ..

I guess it comes down to pricing. The A100 is already ridiculously expensive at $10K. They can this one at $50K and it would sell out?

[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by Uehreka on Tuesday March 22, 2022 @01:57PM (#30769287)
Most people who use one of these will be doing so through an EC2 VM (or equivalent). Given that cloud platforms can spread load, keep these GPUs churning close to 24/7 and more easily predict/amortize costs, they’ll probably buy the amount that they know they need, and Nvidia probably has some approximately correct idea of what that number is.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by p1esk on Tuesday March 22, 2022 @01:51PM (#30769222)
You can buy A100s in a server today, a number of integrators will happily sell it to you.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by chockchocschoir on Tuesday March 22, 2022 @02:15PM (#30769479)
As someone who've tried for some weeks, it really seems like it's out-of-stock literally everywhere. The demand seems to be a lot higher than the supply at the moment, so much that I'm considering buying one myself instead of renting servers with it.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by kovek on Tuesday March 22, 2022 @05:32PM (#30771850)
Does it make sense that all the GPUs are bought out? They each provide a return for mining in the short-term. In the long term, they can be used to run A(G)I models, which will be very very useful
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by fennecfoxen on Tuesday March 22, 2022 @05:38PM (#30771907)
This is the GPU the parent is talking about https://www.nvidia.com/en-us/data-center/a100/
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by kovek on Tuesday March 22, 2022 @05:42PM (#30771955)
This still makes sense! TPUs are useful for AI, which itself will be very very useful. It’s almost like it’s the best investment. That’s why smart players buy them all. Maybe I’m going out-of-topic.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by nqnielsen on Tuesday March 22, 2022 @03:42PM (#30770537)
And even if vendors does say they have it, or can get it, it ended up taking us 4-6 months before systems were online.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by p1esk on Tuesday March 22, 2022 @02:29PM (#30769659)
Did you check Lambda or Exxact?
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by chockchocschoir on Tuesday March 22, 2022 @02:37PM (#30769752)
Yes, nor Lambda Labs or Exxact Corporation have them available last time I checked (last week). Both citing high demand as the reason for it being unavailable.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by sabalaba on Tuesday March 22, 2022 @04:49PM (#30771350)
We (Lambda) have all of the different NVIDIA GPUs in stock ---- can you send a message to [email protected] and check in again with your requirements? We're seeing a lot more stock these days as the supply chain crisis of 2021 comes to an end.
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by chockchocschoir on Tuesday March 22, 2022 @07:29PM (#30772961)
I talked with you (Lambda Labs) just a week ago about the A100 specifically and you said that the demand was higher than the supply, and that people should check once a day or something like that to see if it's available in your dashboard. If you clearly have it available now, please say so outright instead of trying to push some other offer on me in emails :)
[ Parent | on HN ]
Re:Nvidia Hopper GPU Architecture and H100 Accelerator by asciimike on Tuesday March 22, 2022 @05:50PM (#30772024)
Howdy, I run [Crusoe Cloud](https://crusoecloud.com/) and we just launched an alpha of an A100 and A40 Cloud offering--we've got capacity at a reasonable price!

If you're interested in giving us a shot, feel free to shoot me an email at mike at crusoecloud dot com.

[ Parent | on HN ]