News for hackers. Stuff that matters.

A comment on Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more. Parent comment

Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by spiralk on Tuesday September 24, 2024 @01:39PM (#41639000)
The Aider leaderboards seem like a good practical test of coding usefulness: https://aider.chat/docs/leaderboards/. I haven't tried Cursor personally but I am finding Aider with Sonnet more useful that Github Copilot and its nice to be able to pick any model API. Eventually even a local model may be viable. This new Gemini model does not rank very high unfortunately.
[ Parent | on HN ]
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by kendallchuang on Tuesday September 24, 2024 @02:03PM (#41639265)
Thanks for the link. That's unfortunate, though perhaps the benchmarks will be updated after this latest Gemini release. Cursor with Sonnet is great, I'll have to give Aider a try as well.
[ Parent | on HN ]
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by KaoruAoiShiho on Tuesday September 24, 2024 @03:44PM (#41640154)
It's on the leaderboard, it's tied with qwen 2.5 72b and far below SOTA of o1, claude sonnet, and deepseek. (also below very old models like gpt-4-0314 lol)
[ Parent | on HN ]
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by spiralk on Tuesday September 24, 2024 @02:07PM (#41639309)
It is updated actually, gemini-1.5-pro-002 is this new model.
[ Parent | on HN ]
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by kendallchuang on Tuesday September 24, 2024 @04:41PM (#41640696)
That was fast, I missed it!
[ Parent | on HN ]