News for hackers. Stuff that matters.
Re:Ask HN: Best way to deploy an LLM for 100s of user by samspenc on Friday July 05, 2024 @04:40PM (#40885806)
Agreed, this works great for a single or small set of users (2-3), but OP is asking about self-hosting an open-source / open-weight LLM locally to serve a larger set of users (100s per post title). I suspect they may need a larger server farm with a lot of GPUs to handle the load.

There are no comments on this comment yet.