Re:I Run LLMs Locally
by alkonaut on Monday December 30, 2024 @06:06AM
(#42548277)
Yes I get that using the models costs a lot more than training. But using them isn’t free. Even using a large nVidia chip for a fraction of a second is expensive. So where does this money come from? Gains from paid tiers, investor money?
[ Parent
| on HN ]
Re:I Run LLMs Locally
by drillsteps5 on Tuesday December 31, 2024 @12:43AM
(#42556692)
You got it backwards. Training is extremely costly in many different ways, while running inference is much less expensive. Meta, Google, Alibaba, and some others train their models and put them into the open for free burning through their cash reserves trying hard not to lose it all to OpenAI (and MSFT).