If it makes sense to expand scope to provide a particular model server and the group can easily be the best st it, I say go for it. But do it as a separate (but perhaps connected) project to this.

But in general I’m in agreement that this sounds like a separate concept than any given model server.

That said, where is a list of model servers for the most commonly wanted LLMs at this point?

Perhaps maintaining a list of those that do and don’t work with the proxy would be helpful.