Re:Use the Gemini API with OpenAI Fallback in TypeScript
by visarga on Monday April 07, 2025 @02:36AM
(#43608476)
I just had very bad JSON mode operation with gemini-1.5-flash and 2.0-flash models using their own library 'google-generativeai'. Either can't follow JSON formatting correctly, or renders string fields with no end until max_tokens. Pretty bad for Gemini, when open models like Qwen do a better job of a basic information extraction to JSON task.
[ Parent
| on HN ]
Re:Use the Gemini API with OpenAI Fallback in TypeScript
by MrBuddyCasino on Monday April 07, 2025 @07:53AM
(#43610240)
Did you provide a JSON schema? I've had good experience with that.
Re:Use the Gemini API with OpenAI Fallback in TypeScript
by zbrw on Monday April 07, 2025 @09:11AM
(#43610934)
Things to note:
1) supply a JSON schema in `config.reponse_schema`
2) set the `config.response_type` to `application/json`
That works for me reliably. I've had some issues with running into max_token constraints but that was usually on me because I had let it process a large list in one inference call, which would have resulted in very large outputs.
We're using gemini JSON mode in production applications with both `google-generativeai` and `langchain` without issues.