Nvidia Hopper GPU Architecture and H100 Accelerator

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by rafaelero on Tuesday March 22, 2022 @01:42PM (#30769117)

"Combined with the additional memory on H100 and the faster NVLink 4 I/O, and NVIDIA claims that a large cluster of GPUs can train a transformer up to 9x faster, which would bring down training times on today’s largest models down to a more reasonable period of time, and make even larger models more practical to tackle."

Looking good.

[ Parent | on HN ]

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by ml_hardware on Tuesday March 22, 2022 @02:41PM (#30769813)

The 9x speedup is a bit inflated... it's measured at a reference point of ~8k GPUs, on a workload that the A100 cluster is particularly bad at.

When measured at smaller #s of GPUs which are more realistic, the speedup is somewhere between 3.5x - 6x. See the GTC Keynote video at 38:50: https://youtu.be/39ubNuxnrK8?t=2330

Based on hardware specs alone, I think that training transformers with FP8 on H100 systems vs. FP16 on A100 systems should only be 3-4x faster. Definitely looking forward to external benchmarks over the coming months...

[ Parent | on HN ]

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by Melatonic on Tuesday March 22, 2022 @01:44PM (#30769132)

We have needed wide use of NVlink or something like it for a long time now......heres to hoping mobo manufacturers actually widely implement it!

[ Parent | on HN ]

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by learndeeply on Tuesday March 22, 2022 @03:50PM (#30770634)

The open standard version of NVLink is CXL. They're available in latest gen CPUs.

[ Parent | on HN ]

Re:Nvidia Hopper GPU Architecture and H100 Accelerator by Melatonic on Tuesday March 22, 2022 @04:13PM (#30770902)

Interesting - I did not know that. Don't we also need motherboard manufacturers though to more widely implement the hardware required? It has been awhile since I have read about NVlink to be fair

[ Parent | on HN ]