News for hackers. Stuff that matters.
Re:GPT-4 API General Availability by weird-eye-issue on Friday July 07, 2023 @05:54AM (#36629444)
I built a production app on top of OpenAI and yeah there are frequent errors and timeouts but you literally just have to add some code to account for these and it works fine...

For example exponential back off is the first step, then adding retrying on timeouts (we use streaming and if there are 30 seconds in between getting data back we retry the whole request - rare but happens), then fixing anything else that pops up

It is possible to have a stable production app on top of it

Just fix your code and stop expecting OpenAI to hold your hand

Re:GPT-4 API General Availability by jwr on Friday July 07, 2023 @05:59AM (#36629473)
> Just fix your code and stop expecting OpenAI to hold your hand

:-)

My code does retry and the entire application is written to detect and work around breakage. But eventually I do need to get enough content from OpenAI API to be able to make progress, and I am not.

At the moment, for example, all requests just time out after 12 minutes (on my side). No amount of "fixing my code" will help, and I don't want OpenAI to hold my hand, I just want it to a) return some data at least sometimes, b) not charge me for data not delivered.

Let's look at my billing page: over the last hour it shows 8 requests. A total of 52584 tokens. Not a single response made it back to me.

Re:GPT-4 API General Availability by weird-eye-issue on Friday July 07, 2023 @06:08AM (#36629557)
Something is seriously wrong with your network (or your code). That's it that's your answer. It's not OpenAI's fault.

We spend over $20k/mo with them and don't have any issues like this.

12 minute timeouts make no sense because first of all why are you even waiting 12 minutes?

Get off HN and go fix your shit

Re:GPT-4 API General Availability by jwr on Friday July 07, 2023 @06:17AM (#36629613)
I am very happy it works for you! But that does not imply that it works for everyone. Wish you all the best.
Re:GPT-4 API General Availability by weird-eye-issue on Friday July 07, 2023 @06:34AM (#36629717)
The common denominator with these errors is you. Good luck.