News for hackers. Stuff that matters.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by binarymax on Tuesday March 21, 2023 @01:55PM (#35249847)
It is interesting that hopper isn’t widely available yet.

I have seen some benchmarks from academia but nothing in the private sector.

I wonder if they thought they were moving too fast and wanted to milk amphere/ada as long as possible.

Not having any competition whatsoever means Nvidia can release what they like when they like.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by TylerE on Tuesday March 21, 2023 @01:58PM (#35249894)
Why bother when you can get cryptobros paying way over MSRP for 3090s?
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by binarymax on Tuesday March 21, 2023 @02:02PM (#35249957)
Not just cryptobros. A100s are the current top of the line and it’s hard to find them available on AWS and Lambda. Vast.AI has plenty if you trust renting from a stranger.

AMD really needs to pick up the pace and make a solid competitive offering in deep learning. They’re slowly getting there but they are at least 2 generations out.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by fbdab103 on Tuesday March 21, 2023 @02:09PM (#35250075)
I would take a huge performance hit to just not deal with Nvidia drivers. Unless things have changed, it is still not really possible to operate on AMD hardware without a list of gotchas.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by brucethemoose2 on Tuesday March 21, 2023 @02:23PM (#35250293)
Its still basically impossible to find MI200s in the cloud.

On desktops, only the 7000 series is kinda competitive for AI in particular, and you have to go out of your way to get it running quick in PyTorch. The 6000 and 5000 series just weren't designed for AI.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by breatheoften on Tuesday March 21, 2023 @04:36PM (#35252149)
It's crazy to me that no other hardware company has sought to compete for the deep learning training/inference market yet ...

The existing ecosystems (cuda, pytorch etc) are all pretty garbage anyway -- aside from the massive number of tutorials it doesn't seem like it would actually be hard to build a vertically integrated competitor ecosystem ... it feels a little like the rise of rails to me -- is a million articles about how to build a blog engine really that deep a moat ..?

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by wmf on Tuesday March 21, 2023 @05:48PM (#35253058)
There are tons of companies trying; they just aren't succeeding.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by KeplerBoy on Tuesday March 21, 2023 @07:06PM (#35254041)
How could their moat possibly be deeper?

First of all you need hardware with cutting-edge chips. Chips which can only be supplied by TSMC and Samsung.

Then you need the software ranging all the way from the firmware and driver over something analogous to CUDA with libraries like cuDNN, cuBLAS and many others to integrations into pytorch and tensorflow.

And none of that will come for free, like it came to Nvidia. Nvidia built CUDA and people built their DL frameworks around it in the last decade, but nobody will invest their time into doing the same for a competitor, when they could just do their research on Nvidia hardware instead.

Realistically it's up to AMD or Intel.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by rcme on Tuesday March 21, 2023 @07:37PM (#35254418)
There will probably be Chinese options as well. China has an incentive to provide a domestic competitor due to deteriorating relations with the U.S.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by KeplerBoy on Wednesday March 22, 2023 @02:20AM (#35257478)
They certainly will have to try, since nvidia is banned from exporting A100 and H100 chips.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by HTTP418 on Wednesday March 22, 2023 @01:51PM (#35264634)
They do ship A800 and H800 to China. H800 is the H100 with a much slower memory bandwidth. A800 is also a tiered down version of the A100
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by runnerup on Tuesday March 21, 2023 @11:43PM (#35256553)
No other company has sought this?

https://www.cerebras.net/ Has innovative technology, has actual customers, and is gaining a foothold in software-system stacks by integrating their platform into the OpenXLA GPU compiler.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by andy81 on Tuesday March 21, 2023 @03:14PM (#35251076)
GPU mining died last year.

There's so little liquidity post-merge that it's only worth mining as a way to launder stolen electricity.

The bitcoin people still waste raw materials, and prices are relatively sticky with so few suppliers and a backlog of demand, but we've already seen prices drop heavily since then.

Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by TylerE on Tuesday March 21, 2023 @03:17PM (#35251135)
Right, that's why NVidia is acutally trying again. The money printer has run out of ink.
Re:Nvidia Announces H100 NVL – Max Memory Server Card for La... by pixl97 on Tuesday March 21, 2023 @03:18PM (#35251139)
The question is, do they not have much production, or is OpenAI and Microsoft buying every single one they produce?