Re:I Run LLMs Locally
by kolbe on Sunday December 29, 2024 @04:02PM
(#42543075)
I think the bigger issue is hardware utilization. Meta and Qwen's medium sized models are fantastic and competitive with 6 month old offerings from OpenAI and Claude, but you need to have $2500 of hardware to run it. If you're going to be spinning this 24/7, sure the API costs for OpenAI and Anthropic would be insane, but as far as normal personal usage patterns go, you'd probably spend $20/month. So, either spend $2500 that will depreciate to $1000 in two years or $580 in API costs?
[ Parent
| on HN ]
There are no comments on this comment yet.