Re:Jsonformer: A bulletproof way to generate structured outp...
by gamegoblin on Tuesday May 02, 2023 @03:13PM
(#35792659)
I hate that gpt-3.5-turbo is so cheap that using systems like guardrails is a sane thing to do. I can almost always prompt davinci-003 without guardrails in a way to get my exact schema 1-shot, whereas guardrails + 3.5-turbo will often consume 2-4x more tokens, but that still makes it significantly cheaper.
[ Parent
| on HN ]
Re:Jsonformer: A bulletproof way to generate structured outp...
by brigadier132 on Tuesday May 02, 2023 @04:41PM
(#35793879)
The problem people are having is hitting the rate limits for chat gpt.
Re:Jsonformer: A bulletproof way to generate structured outp...
by selcuka on Wednesday May 03, 2023 @08:43PM
(#35809841)
Is there a rate limit for the paid API [1] too?
Re:Jsonformer: A bulletproof way to generate structured outp...
by gamegoblin on Thursday May 04, 2023 @06:14PM
(#35822864)
You will get rate limit errors on all the models if you push hard enough. I always wrap the API with some exponential backoff retries to work around this.