Incidentally, Google is often considered to be extremely good at pretraining, which is just the thing internet data is good for.
It looks as if Carbon often takes more than 240 seconds thinking, which I'd guess appears as a silent connection to Google's load balancers, which then drop what it thinks is a stalled connection.
One hopes that Gemini's serving stack is smart enough to cancel these torn-down connections, but maybe they're not and this is leading to all sorts of ghost TPU load.
Disclaimer: We spend millions on GCP per month and have a dedicated customer engineer. Even they are horrified at the glacial pace of the Gemini fixes.
I think no one tests this as a "non-Google employee" or something, they all have specific testing backends or something, and once deployed to the public, there is no one responsible for the experience of non-Googlers and no one testing it, that's why most of the Google stuff has such poor UX because of hundreds of small cuts all over the experience.
Look how all Chinese models, or haiku/luna class US models, you can see the same pattern: they think too much and only work well with a subset of tasks
If GPT slips meaningfully behind, OpenAI is dead. The same goes for Anthropic. Both would die extremely quickly.
If you're within Google that is not the sense you have about the current context, nor within Meta or ByteDance or Amazon or Microsoft. They certainly view all of this as very important, they don't think they'll die within a year or two if they don't stay on top of the frontier. Meta has run from behind endlessly, as has Google, and their businesses are doing just fine (for now). If OpenAI were in that position, there would be no reason for them to exist and they would be dead by now.
Microsoft likely doubts their existence is at risk just because they can't field a frontier model. For now they don't see Office / Windows being killed off as a short-term risk (plus they have Azure and other products).
AWS is extraordinarily difficult to replicate, it's more akin to trying to compete with dozens of steel mills (industrial infrastructure) than a mere piece of software. And the rest of Amazon's cash spigot is advertising on its retail platform, which is facing close to zero threat from the frontier race, so Amazon can take its time and endlessly screw-up on AI (and so it has).
A frontier LLM can't manufacture a smartphone or laptop for you, and certainly can't manufacture 100 million of them with a zillion dollar manufacturing contract with TSMC et al., Apple isn't particularly worried. They want intelligence on their platform, they're not overly concerned about needing to own a frontier model. They can afford to be patient and get it right.
Capital is more powerful than AI. The capital owns the GPUs the AI requires, the capital owns the datacenters, the capital owns the grid, the capital owns the generation. Apple, Microsoft, Google, Amazon, Nvidia control the capital, financially they're far more powerful than most mega banks these days. And if Apple needed $100 billion for a project, it's one human phone call to Berkshire Hathaway. For now AI is feeble standing next to capital.
2) lots of people inside Google use carbon. If they didn't care about it, it wouldn't have been pushed on all of Google.
3) Deepmind is competing with cloud and others. they are the only Frontier Lab that actually has to compete with other business units within an org for compute. GCP and others want that compute as well.
(Googler, opinions are my own)
2. GP said they don't care about cost, not they don't care about the model
3. Competing for existing compute + being able to assign the projects own capex investment for new compute provides more compute than needing to assign all compute via new capex investment. But it's hard to say if that's actually the type of case or not unless Google wants to tell everyone publicly - so I'd take that point with a grain of salt in either way.
1. Google has huge amounts of cash and compute. Their goal was to train a model that that has the highest quality & frontier performance, not create a model that's efficient to run and serve. They have other models for that, like Hydrogen.
2. Google created Carbon without caring for inference costs. DeepMind is subsidized by the rest of Google's empire. Selling inference is an insignificant part of Google's revenue. But for Anthropic and OpenAI, selling inference is their entire business model. Anthropic and OpenAI have enormous incentive to optimize model serving and inference costs, far more so than Google does.