News for hackers. Stuff that matters.
Re:The AI Crowd Is Mad by habibur on Wednesday February 01, 2023 @08:52AM (#34610377)
I pretty much always wanted to know the cost of running those models, not training those.

That should be order of magnitude less.

Re:The AI Crowd Is Mad by jackmott42 on Wednesday February 01, 2023 @09:01AM (#34610487)
stable diffusion will generate an image in ~30 seconds on an NVidia 3060, so the cost is on the order of cents per image, power use on the order of 170 watts during the generation, or 0.0014166666667 kwh. @ around 5 cents per kwh
Re:The AI Crowd Is Mad by rileymat2 on Wednesday February 01, 2023 @09:04AM (#34610523)
You also have to calculate the life expectancy of the card against 30 seconds of its life as an ongoing cost.
Re:The AI Crowd Is Mad by MagnumOpus on Wednesday February 01, 2023 @09:31AM (#34610949)
Which is also on the order of 0.01c per query (answering 5 million queries in the lifetime of a $500 card).
Re:The AI Crowd Is Mad by smulc on Wednesday February 01, 2023 @09:10AM (#34610601)
Just ask ChatGPT what the cost is? ;)
Re:The AI Crowd Is Mad by npongratz on Wednesday February 01, 2023 @09:19AM (#34610758)
Sure, but just remember that it can't do math, so any mathy answer it gives is just bullshit.