News for hackers. Stuff that matters.

A comment on OpenLLM.

Re:OpenLLM by d4rkp4ttern on Monday June 19, 2023 @07:33AM (#36389814)
Suppose I’ve written code that calls the OpenAI API. Is there some library that helps me easily switch to a local/other LLM. I.e a library that (ideally) provides the same OpenAI interface for several models, or if not then at least the same interface.
[ Parent | on HN ]
Re:OpenLLM by monkeydust on Monday June 19, 2023 @07:36AM (#36389834)
Langchain might be what you need.
[ Parent | on HN ]
Re:OpenLLM by fortydegrees on Monday June 19, 2023 @07:49AM (#36389940)
There's the OpenedAI-API extension for text-generation-webui: https://github.com/oobabooga/text-generation-webui/tree/main...
[ Parent | on HN ]
Re:OpenLLM by bart_spoon on Monday June 19, 2023 @08:13AM (#36390161)
Langchain offers abstraction across many LLMs, including OpenAI’s.
[ Parent | on HN ]
Re:OpenLLM by d4rkp4ttern on Monday June 19, 2023 @08:22AM (#36390240)
Found a couple others that do something like this. Turns out I had bookmarked them a while ago.

https://github.com/r2d4/openlm

https://github.com/hyperonym/basaran

[ Parent | on HN ]
Re:OpenLLM by abrichr on Monday June 19, 2023 @08:35AM (#36390349)
https://github.com/nomic-ai/gpt4all
[ Parent | on HN ]
Re:OpenLLM by chaoyu on Monday June 19, 2023 @12:26PM (#36392940)
OpenLLM plan to provide an OpenAI-compatible API, which allows you to even use OpenAI's python client to talk to OpenLLM, user just need to change to Base URL to point to your OpenLLM server. This feature is working-in-progress.
[ Parent | on HN ]