AI needs so much power that old coal plants are sticking around

Re:AI needs so much power that old coal plants are sticking ... by AlexandrB on Monday February 12, 2024 @06:19PM (#39351931)

Does anyone have data on the relative cost of running AI-enhanced experiences vs. traditional ones? For example, search: what's the increase in cost to Microsoft when you do a Copilot search vs. a plain Bing search? I have a vague hunch that it's a lot more expensive and is being subsidized by the companies in question as they try to grab market share, but I don't have anything to back up this intuition.

My worry is that we're in a bubble where AI is being applied to cases where it's not actually cost-effective. This might distort the market by suppressing non-AI competitors and then lead to some pretty drastic monetization when the bills start to come due.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by vmfunction on Monday February 12, 2024 @06:58PM (#39352389)

From my limited understanding of this AI field, that it cost over million $ to train the model, after that the the query against the model is fast and doesn't cost a lot money. So with this AI/LLM it is more about the upfront cost, and how well the model is train using that millions dollars, and according to some data expert seems to be an art and a science [1]. Some AI experts here please correct me if I'm wrong.

1: https://www.amorphousdata.com/blog/time-series-vs-regression

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by sureglymop on Monday February 12, 2024 @07:30PM (#39352641)

But then isn't the model out of date very quickly? It would need to be constantly retrained with current information...

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by Legend2440 on Monday February 12, 2024 @07:43PM (#39352776)

Yes, but for now you just accept that - ChatGPT had a knowledge cutoff in 2022 until relatively recently.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by saurik on Monday February 12, 2024 @09:59PM (#39353771)

Just because it doesn't cost that much to use the model vs. to train the model doesn't mean it isn't still quite a bit more expensive per query than a keyword search.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by makz on Monday February 12, 2024 @07:24PM (#39352581)

So, layoffs are happening not because AI is replacing those workers, they are doing it to subsidise AI.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by Legend2440 on Monday February 12, 2024 @07:44PM (#39352788)

Neither. Tech companies are wildly profitable and can afford all these GPUs.

Layoffs are because they just don't think they need that many workers.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by beretguy on Tuesday February 13, 2024 @03:05AM (#39355591)

Or they are switching to cheaper labor overseas.

[ Parent | on HN ]

Re:AI needs so much power that old coal plants are sticking ... by bawolff on Monday February 12, 2024 @06:42PM (#39352199)

My intuition is that traditional search probably takes a lot of energy too.

Its too bad we don't have any actual numbers

[ Parent | on HN ]