News for hackers. Stuff that matters.

A comment on Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut. Parent comment

Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by XorNot on Thursday September 08, 2022 @11:16PM (#32774882)
This is pretty big news when you look at the numbers floating around time to train AIs like Stable Diffusion - which is by far the most disruptive change that's happened recently. 150,000 hours just went down to 100,000 hours if we take the conservative estimate.

Not fast enough (not at the price point) but getting there: of course since it's an entirely parallel problem, the real metric is cost-per-model since in reality you'd just buy time from the cloud.

I figure we're going to start seeing some big changes once cost-per-model puts it within reach of the hobbyist. If I today could buy a custom model for say, $1000 - that's accessible to the hobbyist experimentalist.

[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by 01100011 on Friday September 09, 2022 @03:47AM (#32776497)
Obviously perf and efficiency would be terrible, but could you distribute the training of a diffusion model across volunteers' computers? I would think that, for many fangroups, you'd have a lot of folks who would like to customize a diffusion model for their favorite characters and/or art.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by sanxiyn on Friday September 09, 2022 @09:59PM (#32786963)
Likely yes, it already has been done for DALL-E. Have a look at Training Transformers Together for what is involved: https://arxiv.org/abs/2207.03481.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by dannyw on Thursday September 08, 2022 @11:40PM (#32775048)
Even $10k is enough for a hobbyist group or crowd funding.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by Ajedi32 on Friday September 09, 2022 @12:54AM (#32775452)
Crowd funding models seems especially interesting, since in theory you could fully automate the process. Submit the dataset and code you want to train with, then once it's funded the service automatically runs the training and releases the resulting model to backers.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by macrolime on Friday September 09, 2022 @02:25AM (#32775936)
Such a platform should include a cloud to run the model, where backers would share the revenue earned from people using the model through an API.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by ipsum2 on Friday September 09, 2022 @12:35AM (#32775336)
Stable diffusion is on the order of millions of dollars.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by tehsauce on Friday September 09, 2022 @12:39AM (#32775372)
It was about $200k in compute.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by moonshotideas on Friday September 09, 2022 @01:01AM (#32775493)
Is there a breakdown of the costs available to view?
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by homarp on Friday September 09, 2022 @01:21AM (#32775596)
https://twitter.com/emostaque/status/1563870674111832066 says "We actually used 256 A100s for this per the model card, 150k hours in total so at market price $600k"
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by godmode2019 on Friday September 09, 2022 @03:41AM (#32776464)
But they still have those A100s?

I think you have to compare it to how much ETH you could have mined as the opportunity cost of training a model vs other uses of a GPU

[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by l33tman on Friday September 09, 2022 @04:45AM (#32776903)
I think it was donated by AWS or something. This is not a scenario where it makes sense to buy and maintain your own server park of GPUs, when there is a race to the bottom already with cloud-based GPU rentals.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by homarp on Friday September 09, 2022 @06:46AM (#32777657)
Also compare the impact of releasing Stable Diffusion vs 'yet another coin'/web3 announcement.
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by homarp on Friday September 09, 2022 @05:11PM (#32784967)
https://www.forbes.com/sites/kenrickcai/2022/09/07/stability...
[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by justatdotin on Thursday September 08, 2022 @11:18PM (#32774904)
> by far the most disruptive change

that has been disclosed :/

[ Parent | on HN ]
Re:Nvidia Hopper Sweeps AI Inference Benchmarks in MLPerf Debut by colordrops on Thursday September 08, 2022 @11:32PM (#32774985)
It's disruptive because it has been disclosed.
[ Parent | on HN ]