News for hackers. Stuff that matters.
Re:Gemini 3 Pro Model Card by margorczynski on Tuesday November 18, 2025 @07:21AM (#45964600)
If these numbers are true then OpenAI is probably done, Anthropic too. Still, it's hard to see an effective monetization method for this tech and it clearly is eating Google's main pie which is search.
Re:Gemini 3 Pro Model Card by alecco on Tuesday November 18, 2025 @07:45AM (#45964942)
For SWE it is the same ranking. But if Google's $20/mo plan is comparable to the $100-200 plans for OpenAI and Anthropic, yes they are done.

But we'll have to wait a few weeks to see if the nerfed model post-release is still as good.

Re:Gemini 3 Pro Model Card by siva7 on Tuesday November 18, 2025 @08:58AM (#45966044)
I have a few secret prompts to test complex reasoning capabilities of new models (in law and medicine). Gemini (2.5 pro) is by a wide margin behind Anthropic (sonnet 4.5 basic thinking) and Openai (pro model) on my own benchmark and I trust my own benchmark more than public leaderboards. So it's the other way around. Google is trying to catch up where the others are. It just doesn't seem so to some because Google undercuts prices and most people don't have own complex problems with a verified solution to test against (so they could see how bad Gemini is in reality)
Re:Gemini 3 Pro Model Card by alecco on Tuesday November 18, 2025 @10:18AM (#45967365)
This thread is about Gemini 3. It will be interesting to see your benchmark results when it's available later.
Re:Gemini 3 Pro Model Card by Sol- on Tuesday November 18, 2025 @07:24AM (#45964642)
Why? These models just leapfrog each other as time advances.

One month Gemini is on top, then ChatGPT, then Anthropic. Not sure why everyone gets FOMO whenever a new version gets released.

Re:Gemini 3 Pro Model Card by remus on Tuesday November 18, 2025 @07:33AM (#45964793)
I think google is uniquely well placed to make a profitable business out of AI: They make their own TPUs so don't have to pay ridiculous amounts of money to Nvidia, they have a great depth of talent in building models, they've got loads of data they can use for training and they've got a huge existing customer base who can buy their AI offerings.

I don't think any other company has all these ingredients.

Re:Gemini 3 Pro Model Card by spaceman_2020 on Tuesday November 18, 2025 @08:14AM (#45965328)
The bear case for Google was always the business side would cannibalize the AI side. AI makes search redundant which kills the golden goose
Re:Gemini 3 Pro Model Card by gizmodo59 on Tuesday November 18, 2025 @07:38AM (#45964851)
While I don’t disagree that Google is the company you can’t bet against when it comes to AI, saying other companies are done is a stretch. If they have a significant moat then they should be at the top all the time by then which is not the case though.
Re:Gemini 3 Pro Model Card by remus on Tuesday November 18, 2025 @07:41AM (#45964900)
Agreed, too early to write off others entirely. It'll be interesting to see who comes out the other side of the bubble with a working business.
Re:Gemini 3 Pro Model Card by adriand on Tuesday November 18, 2025 @08:00AM (#45965121)
Anthropic has a fairly significant lead when it comes to enterprise usage and for coding. This seems like a workable business model to me.
Re:Gemini 3 Pro Model Card by bootlooped on Tuesday November 18, 2025 @09:07AM (#45966191)
I feel this is a tenuous position though. I find it incredibly easy to switch to Gemini CLI when I want a second opinion, or when Claude is down.
Re:Gemini 3 Pro Model Card by adriand on Tuesday November 18, 2025 @12:12PM (#45969089)
The enterprise sales cycle is often quite long, though, and often includes a lot of hurdles around compliance, legal, etc. It would take a fairly sustained loss of edge before a lot of enterprises would switch once they're hooked into a given platform. It's interesting to me that Sonnet 4.5 still edges Gemini 3 on SWE bench. This seems to bode well for the trajectory that Anthropic is on.
Re:Gemini 3 Pro Model Card by basch on Tuesday November 18, 2025 @08:28AM (#45965574)
ChatGPT's moat is their name and user habit. People who are using it will keep using it. All/most of the products are _good enough_ for the people who already got used to using them, that they arent exploring competitors.

Microsoft has the chance of changing habit the most by virtue of being bundled into business contracts that have companies with policies not allowing any other product in the workplace.

Re:Gemini 3 Pro Model Card by remus on Tuesday November 18, 2025 @09:58AM (#45967044)
> ChatGPT's moat is their name and user habit. People who are using it will keep using it. All/most of the products are _good enough_ for the people who already got used to using them, that they arent exploring competitors.

They have a long way to go to become profitable though. Those users will get less sticky when openAI starts upping their pricing/putting ads everywhere/making the product worse to save money/all of the above.

Re:Gemini 3 Pro Model Card by netdevphoenix on Tuesday November 18, 2025 @08:49AM (#45965905)
> business contracts that have companies with policies not allowing any other product in the workplace.

Elaborate please. Are you saying that MS is forcing customers to make Copilot the only allowed LLM product?

Re:Gemini 3 Pro Model Card by basch on Tuesday November 18, 2025 @09:40AM (#45966741)
Not quite, but in effect.

Microsoft has contracts to provide software to companies. Companies have policies that only provided software and ai is allowed. Ipso facto

Re:Gemini 3 Pro Model Card by mlnj on Tuesday November 18, 2025 @07:41AM (#45964892)
100% the reason I am long on Google. They can take their time to monetize these new costs.

Even other search competitors have not proven to be a danger to Google. There is nothing stopping that search money coming in.

Re:Gemini 3 Pro Model Card by Zigurd on Tuesday November 18, 2025 @08:22AM (#45965463)
The TPU are a key factor. They are the most mature alternative to Nvidia. Only Google cloud, Azure, and AWS enable you to rent their respective AI chips. Out of those three, google is the only one to have a frontier model. So if they have a real advantage they're not exposed to the financial shenanigans propping up neo clouds like Coreweave.
Re:Gemini 3 Pro Model Card by astrange on Tuesday November 18, 2025 @06:27PM (#45973679)
Making your own stuff is definitely not always economically better. Google has the same supplier cost pressure as Nvidia for DRAM, fabs, etc.
Re:Gemini 3 Pro Model Card by redox99 on Tuesday November 18, 2025 @07:42AM (#45964905)
Considering GPT 5 was only recently released, it's very unlikely GPT will achieve these scores in just a couple of months. If they had something this good in the oven, they'd probably left the GPT 5 name to it.

Or maybe Google just benchmaxxed and this doesn't translate at all in real world performance.

Re:Gemini 3 Pro Model Card by blueblisters on Tuesday November 18, 2025 @08:11AM (#45965274)
They do have unreleased Olympiad Gold-winning models that are definitely better than GPT5.

TBD if that performance generalizes to other real world tasks.

Re:Gemini 3 Pro Model Card by Palmik on Tuesday November 18, 2025 @08:01AM (#45965136)
GPT 5 was released more than 3 months ago. Gemini 2.5 was released less than 8 months ago.
Re:Gemini 3 Pro Model Card by sidibe on Tuesday November 18, 2025 @08:15AM (#45965349)
If not this model, Google at some point is going to get and stay ahead just because they have so many more people and compute resources they can throw at many directions while the others have to make the right choices with how they use their resources each time. Took a while to channel their numbers into a product direction but now I don't think they're going to let up
Re:Gemini 3 Pro Model Card by lukev on Tuesday November 18, 2025 @07:41AM (#45964902)
Or else it trained/overfit to the benchmarks. We won't really know until people have a chance to use it for real-world tasks.

Also, models are already pretty good but product/market fit (in terms of demonstrated economic value delivered) remains elusive outside of a couple domains. Does a model that's (say) 30% better reach an inflection point that changes that narrative, or is a more qualitative change required?

Re:Gemini 3 Pro Model Card by ilaksh on Tuesday November 18, 2025 @07:39AM (#45964874)
The only one it doesn't win is SWE bench which it is significantly behind Claude Sonnet. You just can't take down Sonnet.
Re:Gemini 3 Pro Model Card by stavros on Tuesday November 18, 2025 @07:45AM (#45964944)
Codex has been much better than Sonnet for me.
Re:Gemini 3 Pro Model Card by dotancohen on Tuesday November 18, 2025 @07:54AM (#45965055)
On what types of tasks?
Re:Gemini 3 Pro Model Card by svantana on Tuesday November 18, 2025 @08:19AM (#45965416)
One percentage point is not significant, neither in the colloquial nor the scientific sense[1].

[1] Binomial formula gives a confidence interval of 3.7%, using p=0.77, N=500, confidence=95%

Re:Gemini 3 Pro Model Card by happa on Tuesday November 18, 2025 @07:29AM (#45964712)
This may just be bad recollection from my part, but hasn't Google reported that their search business is right now the most profitable it has ever been?
Re:Gemini 3 Pro Model Card by senordevnyc on Tuesday November 18, 2025 @07:30AM (#45964745)
1) New SOTA models come out all the time and that hasn't killed the other major AI companies. This will be no different.

2) Google's search revenue last quarter was $56 billion, a 14% increase over Q3 2024.

Re:Gemini 3 Pro Model Card by margorczynski on Tuesday November 18, 2025 @08:01AM (#45965138)
1) Not long ago Altman and the OpenAI CFO were openly asking for public money. None of these AI companies have actually any kind of working business plan and are just burning investor money. If the investors see there is no winning against Google (or some open Chinese model) the money will dry up.

2) I'm not suggesting this will happen overnight but especially younger people gravitate towards LLM for information search + actively use some sort of ad blocking. In the long run it doesn't look great for Google.

Re:Gemini 3 Pro Model Card by senordevnyc on Tuesday November 18, 2025 @08:55AM (#45965989)
No, you suggested that LLMs are clearly eating google's lunch already, and there's just no evidence of that. Quite the opposite.
Re:Gemini 3 Pro Model Card by paswut on Tuesday November 18, 2025 @07:33AM (#45964796)
I'd love to see anthropic/openai pop. back to some regular programming. the models are good enough, time to invest elsewhere
Re:Gemini 3 Pro Model Card by llm_nerd on Tuesday November 18, 2025 @08:21AM (#45965446)
They're constantly matching and exceeding each other. It's a hypercompetitive space and I would fully expect one of the others to top various benchmarks shortly after. On pretty much every leading release someone does this "everyone else is done! Shut er down" thing and it's growing pretty weird.

Having said that, OpenAI's ridiculous hype cycle has been living on borrowed time. OpenAI has zero moat, and are just one vendor in a space with many vendors, and even incredibly competent open source models by surprise Chinese entrants. Sam Altman going around acting like he's a prophet and they're the gatekeepers of the future is an act that should be super old, but somehow fools and their money continue to be parted.

Re:Gemini 3 Pro Model Card by netdevphoenix on Tuesday November 18, 2025 @08:55AM (#45965994)
This. If I had to put my money on a survivor, it would be Google because it is an established company with existing revenue modules unrelated to AI. Anthropic and OpenAI won't stand alone without external funding