News for hackers. Stuff that matters.
Re:Ollama releases OpenAI API compatibility by simonw on Thursday February 08, 2024 @05:47PM (#39308864)
I want it to be documented.

I'm fine with it emerging as a community standard if there's a REALLY robust specification for what the community considers to be "OpenAI API compatible".

Crucially, that standard needs to stay stable even if OpenAI have released a brand new feature this morning.

So I want the following:

- A very solid API specification, including error conditions

- A test suite that can be used to check that new implementations conform to that specification

- A name. I want to know what it means when software claims to be "compatible with OpenAI-API-Spec v3" (for example)

Right now telling me something is "OpenAI API compatible" really isn't enough information. Which bits of that API? Which particular date-in-time was it created to match?

Re:Ollama releases OpenAI API compatibility by londons_explore on Thursday February 08, 2024 @06:17PM (#39309147)
It's a JSON API... JSON API's tend to be more... 'flexible'.

To consume them, just assume that every field is optional and extra fields might appear at any time.

Re:Ollama releases OpenAI API compatibility by swyx on Thursday February 08, 2024 @06:34PM (#39309300)
and disappear at any time... was a leetle bit unsettled by the sudden deprecation of "functions" for "tools" with only minor apparante benefit
Re:Ollama releases OpenAI API compatibility by nl on Thursday February 08, 2024 @06:51PM (#39309444)
and what does `auto` even mean?
Re:Ollama releases OpenAI API compatibility by athyuttamre on Friday February 09, 2024 @05:56AM (#39313572)
Hey nl — I work at OpenAI on our API. Do you mean `tool_choice="auto"`? If so, it means the model gets to pick which tool to call. The other options are:

- `tool_choice={type: "function", function: {name: "getWeather"}}`, where the developer can force a specific tool to be called. - `tool_choice="none"`, where the developer can force the model to address the user, rather than call a tool.

If you have any other feedback, please feel free to email me at [email protected]. Thanks!

Re:Ollama releases OpenAI API compatibility by athyuttamre on Friday February 09, 2024 @05:55AM (#39313561)
Hey swyx — I work at OpenAI on our API. Sorry the change was surprising, we definitely didn't do a great job communicating it.

To confirm, the `functions` parameter will continue to be supported.

We renamed `functions` to `tools` to better align with the naming across our products (Assistants, ChatGPT), where we support other tools like `code_interpreter` and `retrieval` in addition to `function`s.

If you have any other feedback for us, please feel free to email me at [email protected]. Thanks!

Re:Ollama releases OpenAI API compatibility by londons_explore on Friday February 09, 2024 @06:03AM (#39313613)
Might be a good idea to have API versions for this... Then when someone builds a product against "version 1", they can be sure that new features might be added to version 1, but no fields will be removed/renamed without openai releasing version 2.
Re:Ollama releases OpenAI API compatibility by te_chris on Friday February 09, 2024 @04:43AM (#39313173)
Amen! The lack of decent errors from OpenAI is the most annoying. They'll silently return 400 with no explanation. Let's hope that doesn't catch on.

OpenAI compatible just seems to mean 'you can format your prompt like the `messages` array'.

Re:Ollama releases OpenAI API compatibility by athyuttamre on Friday February 09, 2024 @05:52AM (#39313540)
Hi te_chris — I work at OpenAI and am currently working to improve our error messages. Would you be willing to share more about what errors you find annoying? My email is [email protected] (or feel free to reply here). Thanks!