Re:Show HN: I built an OSS alternative to Azure OpenAI services
by detente18 on Thursday December 14, 2023 @12:09PM
(#38643844)
LiteLLM proxy (100+ LLMs in OpenAI format) is exactly compatible with the OpenAI endpoint. Here's how to call it with the openai sdk:
```
import openai
client = openai.OpenAI(
api_key="anything", # proxy key - if set
base_url="http://0.0.0.0:8000" # proxy url
)
# request sent to model set on litellm proxy,
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
{
"role": "user",
"content": "this is a test request, write a short poem"
}
])
``` import openai
client = openai.OpenAI( api_key="anything", # proxy key - if set base_url="http://0.0.0.0:8000" # proxy url )
# request sent to model set on litellm proxy,
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ { "role": "user", "content": "this is a test request, write a short poem" } ])
print(response)
``` Docs - https://docs.litellm.ai/docs/proxy/quick_start