News for hackers. Stuff that matters.

A comment on Show HN: Open-Source Load Balancer for Llama.cpp. Parent comment

Re:Show HN: Open-Source Load Balancer for Llama.cpp by ritonlajoie on Sunday June 02, 2024 @03:12AM (#40552053)
I stumbled on this yesterday, it seems to have a queue concept.. https://github.com/ParisNeo/ollama_proxy_server
[ Parent | on HN ]

There are no comments on this comment yet.