Gemini 3.0 spotted in the wild through A/B testing

Re:Gemini 3.0 spotted in the wild through A/B testing by gordonhart on Thursday October 16, 2025 @03:43PM (#45609840)

We extensively benchmark frontier models at $DAYJOB and Gemini 2.5 is the uncontested king outside of a few narrow use cases. Tracks with the rumor that Google has the best pretraining and falls short only in tuning/alignment. Eagerly anticipating Gemini 3 as 2.5, while king of the hill, still has lots of room for improvement!

Edit: narrow use cases are roughly "true reasoning" (GPT-5) and Python script writing (the Claudes)

[ Parent | on HN ]

Re:Gemini 3.0 spotted in the wild through A/B testing by baq on Friday October 17, 2025 @01:09AM (#45613475)

I used gemini almost exclusively before gpt5, but gpt5 is much better for tool calling tasks like agentic coding and thus can handle much longer tasks unattended.

[ Parent | on HN ]

Re:Gemini 3.0 spotted in the wild through A/B testing by int_19h on Friday October 17, 2025 @07:03AM (#45615288)

If by "fall short on alignment" you mean "will shut up and do what it's told" then yes, that's true (with some forceful prompting, but much less so than what's needed with ChatGPT, never mind Claude). I would count that as a benefit, though.

[ Parent | on HN ]

Re:Gemini 3.0 spotted in the wild through A/B testing by zaphirplane on Friday October 17, 2025 @05:45PM (#45622480)

This is strange, I don’t know anyone that picks Gemini. My experience is Grok is actually the best

[ Parent | on HN ]