News for hackers. Stuff that matters.
Re:Google Gemini has the worst LLM API by jauntywundrkind on Saturday May 03, 2025 @07:05PM (#43883074)
In general, it's just wild to see Google squander such an intense lead.

In 2012, Google was far ahead of the world in making the vast majority of their offerings intensely API-first, intensely API accessible.

It all changed in such a tectonic shift. The Google Plus/Google+ era was this weird new reality where everything Google did had to feed into this social network. But there was nearly no API available to anyone else (short of some very simple posting APIs), where Google flipped a bit, where the whole company stopped caring about the rest of the world and APIs and grew intensely focused on internal use, on themselves, looked only within.

I don't know enough about the LLM situation to comment, but Google squandering such a huge lead, so clearly stopping caring about the world & intertwingularity, becoming so intensely internally focused was such a clear clear clear fall. There's the Google Graveyard of products, but the loss in my mind is more clearly that Google gave up on APIs long ago, and has never performed any clear acts of repentance for such a grevious mis-step against the open world, open possibilities, against closed & internal focus.

Re:Google Gemini has the worst LLM API by simonw on Saturday May 03, 2025 @08:02PM (#43883350)
With Gemini 2.5 (both Pro and Flash) Google have regained so much of that lost ground. Those are by far the best long-context models right now, extremely competitively priced and they have features like image mask segmentation that aren't available from other models yet: https://simonwillison.net/2025/Apr/18/gemini-image-segmentat...
Re:Google Gemini has the worst LLM API by jasonfarnon on Saturday May 03, 2025 @08:20PM (#43883449)
I think the commenter was saying google squandered its lead ("goodwill" is how I would refer to it) in providing open and interoperable services, not the more recent lead it squandered in AI. I agree with your point that they've made up a lot of that ground with gemini 2.5.
Re:Google Gemini has the worst LLM API by simonw on Saturday May 03, 2025 @08:35PM (#43883528)
Yeah you're right, I should have read their comment more closely.

Google's API's have a way steeper learning curve than is necessary. So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google.

Their permission model is diabolically complex to figure out too - same vibes as AWS, Google even used the same IAM acronym.

Re:Google Gemini has the worst LLM API by PantaloonFlames on Saturday May 03, 2025 @10:14PM (#43883954)
> So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google.

I don't see that dependency. With ANY of the APIs. They're all documented. I invoke them directly from within emacs . OR you can curl them. I almost never use the wrapper libraries.

I agree with your point that the client libraries are large and complicated, for my tastes. But there's no inherent dependency of the API on the library. The dependency arrow points the other direction. The libraries are optional; and in my experience, you can find 3p libraries that are thinner and more targeted if you like.

Re:Google Gemini has the worst LLM API by Aeolun on Saturday May 03, 2025 @09:20PM (#43883716)
I feel like the AWS model isn’t all that hard for most of their API’s. It’s just something you don’t really want to think about.
Re:Google Gemini has the worst LLM API by paul-tharun on Sunday May 04, 2025 @08:27AM (#43886325)
I sometimes feel the complexity is present by design to increase the switching cost. Once you understand it and set it up on a project, you are locked in, as the perceived cost of moving is too high.
Re:Google Gemini has the worst LLM API by xyzzy_plugh on Sunday May 04, 2025 @10:14AM (#43886818)
This is bizarre to read. gRPC is used _widely_ outside Google. I'm not aware of any API that requires you to use gRPC. I've never found their permission model to be complicated at all, at least compared to AWS.
Re:Google Gemini has the worst LLM API by tyre on Saturday May 03, 2025 @09:33PM (#43883774)
Gemini 2.5 Pro is so good. I’ve found that using it as the architect and orchestrator, then farming subtasks and computer use to sonnet, is the best ROI
Re:Google Gemini has the worst LLM API by PantaloonFlames on Saturday May 03, 2025 @10:15PM (#43883956)
You can also farm out subtasks to the Gemini Flash models. For example using Aider, use Pro for the "strong" model and Flash for the weak model.
Re:Google Gemini has the worst LLM API by tyre on Sunday May 04, 2025 @11:02PM (#43891590)
I’ve tried flash. Sonnet with computer use has been a blast. Limited anecdata led me to 2.5 Pro + 3.7 Sonnet, but this all moves so fast that it good to reevaluate regularly.
Re:Google Gemini has the worst LLM API by egamirorrim on Sunday May 04, 2025 @03:14AM (#43885109)
OOI what's your preferred framework for that managing agent/child agents setup?
Re:Google Gemini has the worst LLM API by tyre on Sunday May 04, 2025 @11:00PM (#43891587)
I use Roo Code. It’s very good.
Re:Google Gemini has the worst LLM API by candiddevmike on Saturday May 03, 2025 @09:59PM (#43883881)
The models are great but the quotas are a real pain in the ass. You will be fighting other customers for capacity if you end up needing to scale. If you have serious Gemini usage in mind, you almost have to have a Google Cloud TAM to advocate for your usage and quotas.
Re:Google Gemini has the worst LLM API by chrisheecho on Sunday May 04, 2025 @02:48AM (#43885003)
We have moved our quota system to Dynamic Shared Quota (https://cloud.google.com/vertex-ai/generative-ai/docs/quotas) for 2.0+ models. There are no quotas in DSQ. If you need a guaranteed throughput there is an option to purchase Provisioned Throughput (https://cloud.google.com/vertex-ai/generative-ai/docs/provis...).
Re:Google Gemini has the worst LLM API by egamirorrim on Sunday May 04, 2025 @03:15AM (#43885112)
The thing is that the entry level of provisioned throughput is so high! I just want a reliable model experience for my small Dev team using models through Vertex but I don't think there's anything I can buy there to ensure it.
Re:Google Gemini has the worst LLM API by dist-epoch on Sunday May 04, 2025 @08:18AM (#43886299)
While we are talking about quotas, can you maybe add an easy way of checking how much you've used/got left?

Apparently now you need to use google-cloud-quotas to get the limit and google-cloud-monitoring to get the usage.

VS Code copilot managed to implement the first part, getting the limit using gemini-2.5-pro, but when I asked gemini to implement the second part it said that integrating cloud-monitoring is too complex and it can't do it !!!!

Re:Google Gemini has the worst LLM API by aaronbrethorst on Saturday May 03, 2025 @09:59PM (#43883879)
Hubris. It seems similar, at least externally, to what happened at Microsoft in the late 90s/early 00s. I am convinced that a split-up of Microsoft would have been invigorating for the spin-offs, and the tech industry in general would have been better for it.

Maybe we’ll get a do-over with Google.

Re:Google Gemini has the worst LLM API by caturopath on Saturday May 03, 2025 @10:09PM (#43883928)
I don't understand why Sundar Pichai hasn't been replaced. Google seems like it's been floundering with respect to its ability to innovate and execute in the past decade. To the extent that this Google has been a good maintenance org for their cash cows, even that might not be a good plan if they dropped the ball with AI.
Re:Google Gemini has the worst LLM API by huntertwo on Saturday May 03, 2025 @10:16PM (#43883961)
Everybody’s thinking the same thing. He sucks.
Re:Google Gemini has the worst LLM API by rs186 on Sunday May 04, 2025 @08:04AM (#43886253)
Answer is simple: he keeps cash coming in and stock price rising. You can compare his performance to his predecessors and CEOs at other companies. That does not necessarily make him a "good" leader in your eyes, but good enough to the board.
Re:Google Gemini has the worst LLM API by HDThoreaun on Sunday May 04, 2025 @10:10AM (#43886799)
googles worth 2 trillion dollars off the back of a website. I think investors are so out of their depth with tech that theyre cool with his mediocre performance
Re:Google Gemini has the worst LLM API by caturopath on Tuesday May 06, 2025 @01:03PM (#43907334)
Two websites and an ad business.
Re:Google Gemini has the worst LLM API by shawabawa3 on Sunday May 04, 2025 @10:58AM (#43887076)
Google is the leader in LLMs and self-driving cars, two of the biggest innovation areas in the last decade, so how exactly has it been floundering in its ability to innovate and execute?
Re:Google Gemini has the worst LLM API by caturopath on Tuesday May 06, 2025 @01:03PM (#43907319)
Google isn't "the leader" in LLMs. Despite a huge funnel to get users in, for intentional use they are a distant second place for consumers, fourth place for LLM APIs, and reputationally treated as an underdog to two tiny companies.
Re:Google Gemini has the worst LLM API by harlysparks on Sunday May 04, 2025 @11:32AM (#43887284)
Perhaps you need to first define "innovation" and maybe also rationalize why that view of innovation is the end-all of determining CEO performance. Otherwise you're begging the question here.

Google's stock performance, revenue growth, and political influence in Washington under his leadership has grown substantially. I don't disagree that there are even better CEO's out there, but as an investor, the framing of your question is way off. Given the financial performance, why would you want to replace him?

Re:Google Gemini has the worst LLM API by caturopath on Tuesday May 06, 2025 @12:57PM (#43907262)
I didn't say that innovation was the end-all of determining CEO performance, though producing new products and creating new markets is the angle that tech tends to go for. I mentioned Google's struggles to execute: they have an astoundingly hard time getting shit done compared to the other largest tech companies.

The counterfactual isn't Google having average performance. You're crediting the stock performance, revenue growth, and political influence (don't really agree this last one was a place Google shined over this period) to Sundar's leadership; I think it has a lot more to do with the company he was handed.

Re:Google Gemini has the worst LLM API by harlysparks on Sunday May 04, 2025 @11:28AM (#43887267)
Google's headcount (and internal red tap) grew significantly from 2012 to 2025. You're highlighting the fact that at some point in its massive growth, Google had to stop relentlessly pushing R&D and allocate leadership focus on addressing technical debt (or at least operational efficiency) that was a consequence of that growth.