News for hackers. Stuff that matters.
Re:Use the Gemini API with OpenAI Fallback in TypeScript by visarga on Monday April 07, 2025 @02:36AM (#43608476)
I just had very bad JSON mode operation with gemini-1.5-flash and 2.0-flash models using their own library 'google-generativeai'. Either can't follow JSON formatting correctly, or renders string fields with no end until max_tokens. Pretty bad for Gemini, when open models like Qwen do a better job of a basic information extraction to JSON task.
Re:Use the Gemini API with OpenAI Fallback in TypeScript by MrBuddyCasino on Monday April 07, 2025 @07:53AM (#43610240)
Did you provide a JSON schema? I've had good experience with that.
Re:Use the Gemini API with OpenAI Fallback in TypeScript by zbrw on Monday April 07, 2025 @09:11AM (#43610934)
Things to note: 1) supply a JSON schema in `config.reponse_schema` 2) set the `config.response_type` to `application/json`

That works for me reliably. I've had some issues with running into max_token constraints but that was usually on me because I had let it process a large list in one inference call, which would have resulted in very large outputs.

We're using gemini JSON mode in production applications with both `google-generativeai` and `langchain` without issues.