News for hackers. Stuff that matters.

Ask HN: Python Meta-Client for OpenAI, Anthropic, Gemini and other LLM API-s?

I am looking for a Python library that generalizes over popular LLM API-s without sacrificing full control (need to use tools, control temp, seed, top-p etc).

Simon Willison llm seems close but seems to focus much on the CLI and not the library, also doesn't seem to give precise control for non-OpenAI models (might be wrong here).

( 2 comments | on HN )
Re:Ask HN: Python Meta-Client for OpenAI, Anthropic, Gemini ... by detente18 on Thursday March 07, 2024 @11:31AM (#39631005)
Hey, are you just looking for litellm - https://github.com/BerriAI/litellm

context - i'm the repo maintainer

Re:Ask HN: Python Meta-Client for OpenAI, Anthropic, Gemini ... by jesse_portal on Thursday March 07, 2024 @11:34AM (#39631031)
Yeah, I was also going to recommend litellm. I've been using it for my LLM assistant app and was able to add support for pretty much any model or provider endpoint by switching out one line of code.