News for hackers. Stuff that matters.
Re:Jsonformer: A bulletproof way to generate structured outp... by gamegoblin on Tuesday May 02, 2023 @03:13PM (#35792659)
I hate that gpt-3.5-turbo is so cheap that using systems like guardrails is a sane thing to do. I can almost always prompt davinci-003 without guardrails in a way to get my exact schema 1-shot, whereas guardrails + 3.5-turbo will often consume 2-4x more tokens, but that still makes it significantly cheaper.
Re:Jsonformer: A bulletproof way to generate structured outp... by brigadier132 on Tuesday May 02, 2023 @04:41PM (#35793879)
The problem people are having is hitting the rate limits for chat gpt.
Re:Jsonformer: A bulletproof way to generate structured outp... by selcuka on Wednesday May 03, 2023 @08:43PM (#35809841)
Is there a rate limit for the paid API [1] too?

[1] https://platform.openai.com/docs/models/gpt-3-5

Re:Jsonformer: A bulletproof way to generate structured outp... by gamegoblin on Thursday May 04, 2023 @06:14PM (#35822864)
You will get rate limit errors on all the models if you push hard enough. I always wrap the API with some exponential backoff retries to work around this.