I work in this field. I would love to see what you are basing these assertions off of.

> they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc)

Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.

Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys.

All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.

I can assure you Google BERT isn't vaporware.

It was probably a challenge to integrate it into search, but they did that.

So your assertion has been refuted based on your use of "all", at the very least.

Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.

I’ve been hearing “Google has secret better models” for 7 months now. Maybe some UFOs in the hangers at Moffett Field too?

Would you believe OpenAI has vastly better models that they are not releasing publicly?

No

GPT models were internally available 6-12 months before they've seen public beta, of course OpenAI has more capable internal models.

There's no reason to believe this. The training time and cost is so substantial that they are almost certainly building their next release, but it isn't sitting there rotting.

Much of that training time is RLHF, the absence of which does not make the model less capable of carrying out useful tasks (indeed, in case of GPT-4, it actually made the model slightly less capable).

OpenAI themselves have said they hat GPT-4 internally before they ever released the first version of ChatGPT.

Do you realize that LLaMA-1 is just a very slightly smaller, comparably performing replication of Chinchilla [1], which DeepMind had completed a year prior to LLaMA's release? And has RLHF-ed into a suitable chatbot "Sparrow" [2] months earlier than ChatGPT was launched?

To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models but they've arrived at those not through training and benchmarking but by divination and bullshitting. This, let us say, does not sound plausible.

Then again, Microsoft uses LLaMA for research, and they should theoretically have some ability to get stuff from OpenAI. Evidently this isn't how any of this works, huh.

1. https://arxiv.org/abs/2203.15556

2. https://en.wikipedia.org/wiki/Sparrow_(bot)

Google _internally_ feels that they are way behind. Forget commenters on HN, literally all of the google employees that I know believe that the company is failing here.

This is not responsive to my arguments. Google can be arbitrarily far behind OpenAI or Anthropic, OP's idea that they feel threatened by LLaMA when they (well, Deepmind) have reached LLaMA level 18-10 months ago is still wrong.

Hard disagree. Google has made it plainly clear that they don't have anything useable in this space. Bard scores below all other commercial model.

Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.

Bard is a 4.5B or so model.

> Google has made it plainly clear that they don't have anything useable in this space.

Google hasn't made their best models public because they're too expensive to run for free.

> Google is getting the asses handed to them, badly.

Bard has 30M active users and isn't even available in large parts of the world. They're in 2nd place - when they were pretty late to the game - that's an odd way to say someone is getting their ass handed to them.

> Google hasn't made their best models public because they're too expensive to run for free.

? It's the same with paid models.

I am paying per each request sent to Google Generative AI and this is what I get: https://i.ibb.co/4KCmz55/bard1.png

...

Why do you think Google had even bothered with Bard?

And then, given that, why is it worse than the competition?

It seems you didn't quite hear the argument. I agree with you that the models Google has released to the public are absolutely worthless. That certainly does not mean they don't have extremely performant models at all however.

If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great public facing services, and Meta should be congratulated for releasing these models (although I would still prefer the Galactica training data), but academically Deepmind is one of the best groups around.

On that front, Google's Gemini sounds interesting.

See https://www.tomsguide.com/news/googles-new-gemini-ai-could-b...

It's better to wait and see. Either way, they are scraping everyone and everything. If they can't do it...

Why would they have secret unreleased models?

Surely Google can find another team of code monkeys to whip out a frontend if there is money to be made.

I don't think Google is going to pull back from making some more money.

I think the most likely option is that they have a bunch of talented academics who get paid on time to work on what interest them - but they're the stereotypical large inefficient company and they can't coordinate the effort of productionizing some cool models before the competition.

> but academically Deepmind is one of the best groups around

I think your argument is basically that Google has the potential to create the best models because of superiority in the theory of LLMs, even though we hear of no signs from the board, the ceo, or beta releases or product showcases.

But let’s say you’re right. When do you think we would experience the supremacy of DeepMind in our daily lives?

This reminds me of how any day now their self driving cars are going to work right.

Their self driving cars do work? I rode in one for 30 minutes one-way on Sunday. Used it for my return trip too. No driver. Take at least 2 - 3 rides a week and have been for a few months now.

They work (most of the time) in Phoenix and SF because they've mapped every single inch of the cities by now and there are no adverse conditions. It's not scalable.

Why is that not scalable? Mapping out two large cities for an experimental project in a few years seems scalable, expand to new cities over time with additional resources.

I suppose it is, but not in a Silicon Valley way. They could scale to "large Southwestern city taxi service," but it wouldn't earn back the investment or deliver on the hype. If that becomes the ceiling I bet Google will simply shut Waymo down.

If they work out how to deal with, say, New York weather conditions, there's potential, but they don't seem to be any closer.

I think you’re conflating doable and scalable.

Or perhaps my threshhold for “scalable” takes different parameters and weigh these inputs differentfly from you.

Lol google saying they have better models in private is like that one kid that insists he has an awesome girlfriend, but 'she goes to another school, you wouldn't know her'.

I'm pretty sure if google had something much better, the board and C-suite execs would have at least ensured we saw previews of it by now...

Source?