News for hackers. Stuff that matters.

A comment on H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark. Parent comment

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by reaperman on Tuesday June 27, 2023 @07:13PM (#36500013)
It’s remarkable that one of these could train GPT-3 in under a month. However, good to note that its the price is that of twenty 4090’s.
[ Parent | on HN ]
Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by ClassyJacket on Tuesday June 27, 2023 @08:14PM (#36500550)
Could it actually? Or do you need the memory size of thousands of them running in parallel?
[ Parent | on HN ]
Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by reaperman on Tuesday June 27, 2023 @09:44PM (#36501249)
Yes I worded that extremely poorly.
[ Parent | on HN ]
Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by cavisne on Tuesday June 27, 2023 @08:28PM (#36500658)
One interesting thing is if the model can’t fit into GPU memory (sharded across multiple chips) it would be much slower. So one chip would take more than a month.

That’s why these clusters are so valuable, even with Nvidias margins they are still cheaper than using less compute for longer.

[ Parent | on HN ]