Yes I get that using the models costs a lot more than training. But using them isn’t free. Even using a large nVidia chip for a fraction of a second is expensive. So where does this money come from? Gains from paid tiers, investor money?
Re:I Run LLMs Locally
by drillsteps5 on Tuesday December 31, 2024 @12:43AM
(#42556692)
You got it backwards. Training is extremely costly in many different ways, while running inference is much less expensive. Meta, Google, Alibaba, and some others train their models and put them into the open for free burning through their cash reserves trying hard not to lose it all to OpenAI (and MSFT).
The same applies (to smaller extent) to Google, and more recently to Alibaba.
A lot of free (not open-source, mind you) LLMs are either from Meta or built by other smaller outfits by tweaking Meta's LLMs (there are exceptions, like Mistral).
One thing to keep in mind is that while training a model requires A LOT of data, manual intervention, hardware, power, and time, using these LLMs (inference, or forward pass in the neural network) is really not. Right now it typically requires NVidia GPU with lots of VRAM but I have no doubt in just a few months/maybe a year someone will find much easier way to do that without sacrificing the speed.