H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by lumost on Tuesday June 27, 2023 @11:05PM (#36501760)

This points to an interesting future for foundation models. This is an 18x cost reduction in only 2 years. Either foundation models are going to get much bigger, or variations will become common.

[ Parent | on HN ]

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by rerx on Wednesday June 28, 2023 @02:52AM (#36503214)

V100 GPUs are from 2017, so it's more than two years. A100 already appeared there years ago, btw.

An eight GPU DGX-1 server cost ~149k$ back then (googled news postings). A current gen DGX H100 is 520k$ with 5 years of support. Of course it holds 5x the memory, plus GPUs and interconnect are much faster. But when comparing costs, take price hikes into account.

[ Parent | on HN ]

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by jsjohnst on Wednesday June 28, 2023 @09:02AM (#36505804)

An important thing to also keep in mind is how much inflation changed prices over the duration. $520k in 2023 dollars is around $420k in 2017 dollars. Sure, still almost 3x more expensive, but that’s better than being 0.7x higher.

[ Parent | on HN ]

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by raverbashing on Wednesday June 28, 2023 @04:03AM (#36503670)

Variations of specializations I guess

For writing code you don't care about feeding world history to your model. So a smaller model might be better at a specialized task

Sure, having a big multi-modal-model is great, but by having specialized models you can spread tasks better

[ Parent | on HN ]

Re:H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark by mlboss on Wednesday June 28, 2023 @10:04AM (#36506657)

But I am sure prompt understanding improves with more text data. Same with reasoning ability.

[ Parent | on HN ]