Swap OpenAI with any open-source model

Re:Swap OpenAI with any open-source model by abhpro on Wednesday December 06, 2023 @01:42PM (#38547890)

This is cool, as a web developer I'm wondering if I should use it to build a chat interface that lets a user instantly swap models, maybe even mid conversation, and use their own API keys. Not sure if something like that already exists.

[ Parent | on HN ]

Re:Swap OpenAI with any open-source model by dragonwriter on Wednesday December 06, 2023 @02:39PM (#38548484)

oobabooga/text-generation-webui already supports this for local models (but doesn't support other hosted models.)

SillyTavern technically supports this, though its not a central use case, so the UI isn't particular convenient for mid-conversation change of models, but it works.

But I don't know of anything where it is a central feature and convenient (and if it was, you could also implement things like parallel requests and choosing which response to keep.)

[ Parent | on HN ]

Re:Swap OpenAI with any open-source model by smarvin2 on Wednesday December 06, 2023 @01:47PM (#38547941)

I'm not sure if something like that already exists, but you could absolutely build it with this! Feel free to jump into our discord with any questions you have

[ Parent | on HN ]