News for hackers. Stuff that matters.
Re:Free data-center heat is allegedly saving a struggling pu... by nl on Thursday March 16, 2023 @09:20PM (#35191492)
> However, with LLMs coming, I could see data center energy usage jump 10x over the next decade.

Just a note: we are probably right now at peak inefficiency for LLMs. There are obvious improvements available but they will only be explored by organizations which have different time vs resources trade-off calculations compared to OpenAI and Google.

Right now OpenAi and Google are racing to ship products. A working training framework is more important than saving $X (where X is a very large number) because capturing marketshare is seen as vital and they have no shortage of capital.

But the massive and rapid progress seen by projects like llama-cpp show how much performance they are leaving on the table. (And yes this also applies at training time).

There are no comments on this comment yet.