News for hackers. Stuff that matters.
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by neha_n on Thursday July 27, 2023 @02:10AM (#36889561)
This is much needed. For someone looking to quickly implement, having a simple interface goes a long way.
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by detente18 on Thursday July 27, 2023 @12:47PM (#36896360)
Thanks
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by yding on Thursday July 27, 2023 @03:25AM (#36890114)
Very cool Ishaan!
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by ij23 on Thursday July 27, 2023 @03:17PM (#36898726)
Thank you !
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by hardware2win on Thursday July 27, 2023 @08:23AM (#36892376)
>completion(..., azure=True)

Why like this?

Re:Show HN: Litellm – simple library to standardize OpenAI, ... by detente18 on Thursday July 27, 2023 @09:11AM (#36892889)
azure uses the openai python sdk, just remaps certain components. The models are also user-named. This makes it hard to detect if a model passed in is an azure model
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by ij23 on Thursday July 27, 2023 @03:21PM (#36898778)
azure models have custom names - eg I call mine 'chat-gpt-test1', I require some flag to know if it's an azure model
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by d4rkp4ttern on Thursday July 27, 2023 @08:30AM (#36892443)
Great start! Are you planning to add to following:

Retries w exponential backoff, Caching, Streaming output, Function-calling support

Re:Show HN: Litellm – simple library to standardize OpenAI, ... by detente18 on Thursday July 27, 2023 @09:09AM (#36892872)
You can use tenacity for retries and wouldn't you want to cache the request / response around the endpoint instead of the gpt call -> that's what we ended up doing.

Streaming output and function-calling support is interesting

Re:Show HN: Litellm – simple library to standardize OpenAI, ... by d4rkp4ttern on Saturday August 05, 2023 @11:36PM (#37018764)
Could you elaborate on this — “wouldn’t want to cache around the endpoint instead of GPT call”. Just want to see if I’m missing an important consideration here
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by detente18 on Saturday August 12, 2023 @05:48PM (#37104544)
You're probably calling openai from a backend server. If you're making the call because a user asked a question -> your endpoint received it -> did some processing -> called openai -> returned the response,

why would you cache the openai call instead of the endpoint that's receiving the user call?

Re:Show HN: Litellm – simple library to standardize OpenAI, ... by ij23 on Thursday July 27, 2023 @03:46PM (#36899124)
good points, probably going to add streaming output, function calling support. As for retries tenacity does a great job already
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by kaushik92 on Thursday July 27, 2023 @12:38PM (#36896206)
This is amazing. Really needed something like this to standardize all my different AI APIs!

On a side note - I love how quickly your team is shipping! Do keep it going!

Re:Show HN: Litellm – simple library to standardize OpenAI, ... by detente18 on Thursday July 27, 2023 @12:47PM (#36896359)
Thanks!
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by ij23 on Thursday July 27, 2023 @03:16PM (#36898723)
Thanks!
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by uripeled2 on Thursday July 27, 2023 @05:00PM (#36900108)
Take a look at llm-client a similar library that also support chat, async and more llm providers https://github.com/uripeled2/llm-client-sdk
Re:Show HN: Litellm – simple library to standardize OpenAI, ... by ij23 on Thursday July 27, 2023 @06:26PM (#36900965)
thanks for sharing, while your library looks really powerful my goal with Litellm is simplicity