News for hackers. Stuff that matters.
Re:Ask HN: Recommendations for Local LLMs in 2024: Private a... by FezzikTheGiant on Monday April 08, 2024 @07:10PM (#39974637)
I asked this another place in this thread, but curious to know how you plan to run and deploy this model. What's the cheapest or I guess most cost efficient way to do this without burning money. I'm a college student for reference so I don't have a lot of money lying around to experiment.
Re:Ask HN: Recommendations for Local LLMs in 2024: Private a... by malux85 on Monday April 08, 2024 @08:22PM (#39975108)
You’re right it’s expensive,

I have infra in my house, not gonna lie it cost a lot, I have a rack with 30k of equipment in it (including 5 GPUs)

But this would probably run on an AWS P2 instance with is 0.90 USD an hour, or there’s lambda labs which is also pretty cheap (no affiliation, just satisfied customer)

Re:Ask HN: Recommendations for Local LLMs in 2024: Private a... by FezzikTheGiant on Monday April 08, 2024 @09:23PM (#39975452)
Thanks for the info! Appreciate it