I think this is effectively an Apple + Amazon + Google ban?
(MS employee, just noticing interesting intersection of announcements and licensing).
Not that the two situations are anything alike, but a "and look what happened with that" argument hardly points away from valid antitrust outcomes.
I’m not sure you want to invite the comparison.
(Human population / 700 million ≈ 8.75%. Firefox global market share, I've seen measurements reported from 2.81% to 7.69%).
It's essentially a "Amazon and Google don't use this k thx."
TikTok has 1 billion monthly active users for instance
Just above 700m MAU. So yeah, probably aimed at their direct competitors in social.
Apple is the big laggard in terms of big tech and complex neural network models.
v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).
https://github.com/facebookresearch/llama/blob/main/LICENSE
Just like Google scrapes the internet to improve their models, it might make sense to ingest outputs from other models to improve their models. This licensing prevents them from doing that. Using Llama to improve other LLMs is specifically forbidden, but Google will also be forbidden from using Llama to improve any other AI products they might be building.
I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.
It’s not.
Whose goal is that?
And they want to be very careful about labeling outputs as derivative works, because the moment they do that then they have no defense against the model being a derivative work of every single input.
> I wouldn't be surprised if Amazon does as well.
I would - they are not a very major player in this space.
TikTok also meets this definition and probably doesn't have LLM.
It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google.
If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.
> they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc)
Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
It was probably a challenge to integrate it into search, but they did that.
So your assertion has been refuted based on your use of "all", at the very least.
To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models but they've arrived at those not through training and benchmarking but by divination and bullshitting. This, let us say, does not sound plausible.
Then again, Microsoft uses LLaMA for research, and they should theoretically have some ability to get stuff from OpenAI. Evidently this isn't how any of this works, huh.
Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.
Google hasn't made their best models public because they're too expensive to run for free.
> Google is getting the asses handed to them, badly.
Bard has 30M active users and isn't even available in large parts of the world. They're in 2nd place - when they were pretty late to the game - that's an odd way to say someone is getting their ass handed to them.
? It's the same with paid models.
I am paying per each request sent to Google Generative AI and this is what I get: https://i.ibb.co/4KCmz55/bard1.png
...
And then, given that, why is it worse than the competition?
If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great public facing services, and Meta should be congratulated for releasing these models (although I would still prefer the Galactica training data), but academically Deepmind is one of the best groups around.
See https://www.tomsguide.com/news/googles-new-gemini-ai-could-b...
Surely Google can find another team of code monkeys to whip out a frontend if there is money to be made.
I don't think Google is going to pull back from making some more money.
I think the most likely option is that they have a bunch of talented academics who get paid on time to work on what interest them - but they're the stereotypical large inefficient company and they can't coordinate the effort of productionizing some cool models before the competition.
I think your argument is basically that Google has the potential to create the best models because of superiority in the theory of LLMs, even though we hear of no signs from the board, the ceo, or beta releases or product showcases.
But let’s say you’re right. When do you think we would experience the supremacy of DeepMind in our daily lives?
If they work out how to deal with, say, New York weather conditions, there's potential, but they don't seem to be any closer.
Or perhaps my threshhold for “scalable” takes different parameters and weigh these inputs differentfly from you.
I'm pretty sure if google had something much better, the board and C-suite execs would have at least ensured we saw previews of it by now...
Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
https://imagen.research.google/
https://google-research.github.io/seanet/soundstorm/examples...
https://google-research.github.io/seanet/musiclm/examples/
Why would it be surprising that they have better models for resarch that they don't want to give out yet?
The other things you are describing are just standard for research paper releases.
Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz.
Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype.
Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
Not going to keep replying, believe what you want about Google's capabilities
You seem to be under the mistaken belief that: 1. Google has competent high-level organization that effectively sets and pursues long term goals. 2. There is some advantage to developing a highly capable LLM but not releasing it.
(2) could be the case if Google had built an extremely large model which was too expensive to deploy. Having been privy to what they had been working on up until mid-2022 and knowing how much work, compute and planning goes into extremely large models, this would very much surprise me.
Note: I did not have much visibility into what deepmind was up to. Maybe they had something.
> Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L.
https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000...
If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already.
Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
Gotta put products in the market, or it didn't happen...
It seems it is not aware of the notion of historic development, perhaps its world-model is "static"?
Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News".
REF: https://www.stefanjudis.com/today-i-learned/property-order-i...
They will soon be able to train an LLM because it simply has become commoditized, but they just are not a major player in this space at all.
I thought the ml work they do in photos for text selection and facial recognition is pretty neat.
I don’t think we will “hear” about Apple using LLMs either way because they will no doubt call it something different like they always have.
That said, they seem to prefer catchup waiting till others explore new tech they swoop in an (claim) to perfect it from a usability pov. I have no reason to suspect they won’t do the same here.
I agree that there is an incentive to put AI models on your OS. I just don't think Apple can own the whole stack if they want to play ball right now.
While this model in particular isn't likely to land in iOS or Windows, the license definitely suggests Meta is trying to cut off FAANG at the pass.
poor reading of the numbers. one guy at a bank pulled up similarweb and guesstimated 100m registered users and it went viral. whisper numbers were closer to 50m. but in the 6 months since they have certainly crossed 100m and probably are north of 500m, and only recently dipped.
e: nvm. Apple not there either.
But only existing competitors. If you don't yet have 700MM MAU, the impact of this is only that, after you reach 700MM MAU, you can't get future versions of the Llama models for free. You can still continue to use versions that were released before you reached that threshold.
For reference, neither Instagram nor WhatsApp had 700MM MAU at the time Facebook decided to acquire them.
Hmm. Sounds like specifically a FAANG ban. I personally don't mind. But would this be considered anti-competitive and illegal? Not that Google/MS/etc. don't already have their own LLMs.
[1]: https://opensource.com/article/19/4/fauxpen-source-bad-busin...
I suggest we start doing a similar thing for FOSS projects. I.e., it's open source unless you are MegaCorp with profit > $1B, then you have to pay. Sounds fair to me, and might help FOSS to get some better funding.
If it bothers you, you can call it FOSS* instead.
You can always make a commercial closed source project.
Although I suppose they might have an aim of growing past 700m and therefore be deterred.
[1] https://www.statista.com/statistics/303681/twitter-users-wor...
Maybe we should give it a good name and hopefully seeing OSS adopt this.
E.g. 100k/year for each trillion in market cap, updated yearly. First trillion is free
Back in 2011 at my (failed) startup we had a license like that with an NLP provider/ vendor. I don't remember the exact details, but, one caveat was that we HAD to have an equity part to the deal to satisfy Delaware Law/ some other esoteric bit I don't recall.
We already had negotiated an equity stake baked into the agreement, but I do recall that being a very specific requirement both our lawyers raised. I wonder how you could scale something like your proposed scenario and the overhead for the equity requirement, for open source projects.
One way or another it will be tricked.
The limit here is about 10% of the worlds population.
Goods A and B are economic complements if, when the price of A goes down, demand for B goes up.
LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up".
This is a case of commoditizing the competition but that's not the same thing.
Commoditizing your complement:
- All-inclusive resorts. Restaurants are a complement to hotels. If food is free I might go on vacation more.
- Smartphone app-stores. Apps are a complement to phones. If apps cost $0.99 there will be more demand for iphones than if apps cost $20.
This is Zuck being an absolute shark and not wanting his competitors to have a monopoly over LLMs in case they win at some other game. It has nothing to do with "commoditize your complement."
Tell that to the people generating text for social media campaigns using LLMs.
I do think Meta probably benefits from commodity NLP inference as well, but not as a complement.
Zuck is a smart leader. Metaverse was a debacle. But the new world (AI centric) is for real. He is likely focusing on both weakening the stronghold of Google and building a massive community (like Android) around llama. Product ideas (including enterprise focus) will emerge over time.
> If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
Looks like they are trying to block out competitors, it's the perfect commoditize your complement but don't let your actual competitors try to eke out any benefit from it.