News for hackers. Stuff that matters.
Re:Ask HN: Best way to deploy an LLM for 100s of user by gardnr on Friday July 05, 2024 @08:19AM (#40882055)
Not enough info.

Do they want near-realtime responses? Will they all hit it at the same time? Can you put some workloads in an overnight batch queue?

There are no comments on this comment yet.