Show HN: Open-Source Load Balancer for Llama.cpp

Re:Show HN: Open-Source Load Balancer for Llama.cpp by friendly_chap on Sunday June 02, 2024 @04:10PM (#40556892)

I have actually open sourced one recently which supports queues - its both a desktop app and a daemon: https://github.com/singulatron/singulatron

[ Parent | on HN ]

Re:Show HN: Open-Source Load Balancer for Llama.cpp by 3abiton on Sunday June 02, 2024 @05:08PM (#40557210)

So many interesting projects, I just wish AI hardware was readily available.

[ Parent | on HN ]