News for hackers. Stuff that matters.

A comment on Mixtral 8x22B. Parent comment

Re:Mixtral 8x22B by lubesGordi on Wednesday April 17, 2024 @02:35PM (#40068517)
Hey can you guys elaborate how this works? I'm looking at the Ollama section in their docs and it talks about load balancing? I don't understand what that means in this context.
[ Parent | on HN ]
Re:Mixtral 8x22B by SushiHippie on Wednesday April 17, 2024 @11:16PM (#40072627)
You probably want to look at the getting started guide

https://docs.openwebui.com/getting-started/

IIUC the load balancing page is for people who want to run openwebui at a larger scale

https://en.wikipedia.org/wiki/Load_balancing_(computing)

[ Parent | on HN ]