News for hackers. Stuff that matters.
Re:Meta AI releases Code Llama 70B by bk146 on Monday January 29, 2024 @01:07PM (#39179799)
Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand.

(Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

Re:Meta AI releases Code Llama 70B by pchristensen on Monday January 29, 2024 @01:16PM (#39179948)
Meta doesn't have an AI "product" competing with OpenAI, Google's Bard, etc. But they use AI extensively internally. This is roughly a byproduct of their internal AI work that they're already doing, and fostering open source AI development puts incredible pressure on the AI products and their owners.

If Meta can help prevent there from being an AI monopoly company, but rather an ecosystem of comparable products, then they avoid having another threatening tech giant competitor, as well as preventing their own AI work and products from being devalued.

Think of it like Google releasing a web browser.

Re:Meta AI releases Code Llama 70B by IshKebab on Monday January 29, 2024 @01:35PM (#39180317)
Google releasing a (very popular) web browser gives them direct control of web standards. What does this give Facebook?
Re:Meta AI releases Code Llama 70B by fngjdflmdflg on Monday January 29, 2024 @04:37PM (#39183034)
Web standards are probably the last thing Google cares about with Chrome. Much more important is being the default search engine and making sure data collection isn't interrupted by a potential privacy minded browser.
Re:Meta AI releases Code Llama 70B by eganist on Monday January 29, 2024 @01:44PM (#39180479)
OP already mentioned that it adds additional hurdles for possible future tech giants to have to cross on their quest.

It's akin to a Great Filter, if such an analogy helps. If Meta's open models make a company's closed models uneconomical for others to consume, then the business case for those models is compromised and the odds of them growing to a size where they can compete with Meta in other ways is mitigated a bit.

Re:Meta AI releases Code Llama 70B by patapong on Monday January 29, 2024 @01:54PM (#39180665)
I think we should not underestimate the strategic talent acquisition value as well. Many top-tier AI engineers may appreciate the openness and choose to join meta, which could be very valuable in the long run.
Re:Meta AI releases Code Llama 70B by jwkane on Monday January 29, 2024 @02:28PM (#39181186)
Excellent point -- goodwill in a hyper-high demand dev community is invaluable.
Re:Meta AI releases Code Llama 70B by gen220 on Monday January 29, 2024 @01:14PM (#39179917)
They're commoditizing the ability to generate viral content, which is the carrot that keeps peoples' eyeballs on the hedonic treadmill. More eyeball-time = more ad placements = more money.

On the advertiser side, they're commoditizing the ability for companies to write more persuasively-targeted ads. Higher click-through rates = more money.

[edit]: For models that generate code instead of content (TFA), it's obviously a different story. I don't have a good grip on that story, beyond "they're using their otherwise-idle GPU farms to buy goodwill and innovate on training methods".

Re:Meta AI releases Code Llama 70B by esafak on Monday January 29, 2024 @02:34PM (#39181280)
That stuff ultimately drives people away. Who thinks "I need my daily fix of genAI memes, let me head to Facebook!"?
Re:Meta AI releases Code Llama 70B by satvikpendem on Tuesday January 30, 2024 @05:03AM (#39188259)
People on HN are not representative of the average Facebook user.
Re:Meta AI releases Code Llama 70B by Lerc on Monday January 29, 2024 @01:22PM (#39180056)
If they hadn't opened the models the llama series would just be a few sub-GPT4 models. Opening the models has created a wealth of development that has built upon those models.

Alone, it was unlikely they would become a major player in a field that might be massively important. With a large community building upon their base they have a chance to influence the direction of development and possibly prevent a proprietary monopoly in the hands of another company.

Re:Meta AI releases Code Llama 70B by eurekin on Monday January 29, 2024 @01:10PM (#39179852)
Total speculation: Yann LeCun is there and he is really passionate about the technology and openness
Re:Meta AI releases Code Llama 70B by observationist on Monday January 29, 2024 @01:21PM (#39180035)
The faux-open models mean the models can't be used in competing products. The open code base means enthusiasts and amateurs and other people hack on Meta projects and contribute improvements.

They get free R&D and suppress competition, while looking like they have principles. Yann is clueless about open source principles, or the models would have been Apache or some other comparably open license. It's all ruthless corporate strategy, regardless of the mouth noises coming out of various meta employees.

Re:Meta AI releases Code Llama 70B by importantbrian on Monday January 29, 2024 @02:18PM (#39181035)
Meta's choice of license doesn't indicate that Yann is clueless about open-source principles. I don't know about Meta specifically, but in most companies choosing the license for open source projects involves working with a lot of different stakeholders. He very easily could have pushed for Apache or MIT and some other interest group within Meta vetoed it.
Re:Meta AI releases Code Llama 70B by sangnoir on Monday January 29, 2024 @03:40PM (#39182203)
> The faux-open models mean the models can't be used in competing products.

Just because certain entities can't profitably use a product or obtain a license doesn't make it not-open. AGPL is open, for an extreme example.

This argument is also subjective, and not new - "Which is more open BSD-style licenses or GPL?" has ben a guaranteed flameware starter for decades.

Re:Meta AI releases Code Llama 70B by observationist on Monday January 29, 2024 @06:40PM (#39184307)
I'm not arguing about BSD or GPL. I'm saying that the "open source code, proprietary binary blob" pattern Meta is running with is about quashing potential competition, market positioning, and corporate priorities over any tangential beneficial contributions to open source AI.

It's shitty when other companies do it. It's shitty when Broadcom does it. It's shitty when Meta does it.

It's never a not shitty thing to do.

Re:Meta AI releases Code Llama 70B by skottenborg on Monday January 29, 2024 @01:32PM (#39180265)
I doubt personal passions would merit the company funding required for such big models.
Re:Meta AI releases Code Llama 70B by eurekin on Monday January 29, 2024 @01:50PM (#39180601)
Given how megacorps spend millions on a whim (Disney with all recent flops) or, when just a single person wants it (Ms Flight Simulator?) - I wouldn't be surprised to be honest...

But sure, sounds more reasonable

Re:Meta AI releases Code Llama 70B by og_kalu on Monday January 29, 2024 @02:26PM (#39181141)
Disney didn't spend millions on a whim. It's just the reality of box office that even millions in investment are no guarantee for returns.
Re:Meta AI releases Code Llama 70B by eurekin on Tuesday January 30, 2024 @01:24AM (#39186975)
Financially, they have underperformed significantly over longer period of time (10 years):

  For shareholders, this subpar performance has destroyed value. Disney stock has underperformed the stocks
  of Disney’s self-selected proxy peers and the broader market over every relevant period during the last
  decade and during the tenure of each non-management director. Furthermore, it has underperformed since
  Bob Iger was first appointed CEO in 2005 – a period during which he has served as CEO or Executive
  Chairman (directing the Company’s creative endeavors in this role) for all but 11 months. Disney shareholders
  were once over $200 billion wealthier than they are now
Which is radically different from previous 90 years

https://trianpartners.com/wp-content/uploads/2023/12/Trian-N...

Re:Meta AI releases Code Llama 70B by og_kalu on Tuesday January 30, 2024 @11:02AM (#39191685)
Share price isn't the be all end all.

Disney has steamrolled Hollywood for the last decade, bringing in by far the biggest global box office revenue in 7 consecutive years out of 8. They have more billion dollar box office movies than every other studio co mbined. This kind of dominance was unheard of in the history of Hollywood.

Setting box office aside, Disney revenue has tripled since Iger took over and is twice as much as it should be adjusted for inflation.

The idea that the company has underperformed for the last 10 years or that they spend millions "on a whim" is a joke. And using share price as some justification is even more absurd, share price was double what it was today just in 2021.

Re:Meta AI releases Code Llama 70B by eurekin on Tuesday January 30, 2024 @02:24PM (#39194449)
> The idea that the company has underperformed for the last 10 years ... is a joke

Did you even read the Triad Partners quote from their letter? It's their words, not mine.

Re:Meta AI releases Code Llama 70B by og_kalu on Tuesday January 30, 2024 @04:47PM (#39196320)
There is no quote that says that.

"Earnings per share (“EPS”) in the most recent fiscal year were lower than the EPS generated by Disney a decade ago"

is not the same as "underpeforming for a decade".

all that says is that EPS is currently low, not that it has been low and reducing/stagnant for a decade.

Re:Meta AI releases Code Llama 70B by eurekin on Wednesday January 31, 2024 @02:00AM (#39200860)
Ok, it's quite possible I just didn't understand the writing. How about this part?

> Disney shareholders were once over $200 billion wealthier than they are now

Is that an exaggeration?

Re:Meta AI releases Code Llama 70B by eurekin on Wednesday January 31, 2024 @04:23AM (#39201749)
From here:

https://filmthreat.com/news/male-and-pale-is-stale-responses...

  As a result, the money generated by the industry has seen a 
  remarkably noticeable drop. The rise of the term “flop buster”
  (in response to so many films like Indiana Jones and the 
  Dial of Destiny underperforming at the box office) seems to 
  be directly related to mainstream audiences’ wholesale rejection
  of the over-messaging they see hijacking entertainment.
addresses your "it's just nature of Box Office" flopping argument
Re:Meta AI releases Code Llama 70B by chasd00 on Monday January 29, 2024 @02:38PM (#39181332)
My opinion is Meta is taking the model out of the secret sauce formula. That leaves hardware and data for training as the barrier to entry. If you don't need to develop your own model then all you need is data and hardware which lowers the barrier to entry. The lower the barrier the more GenAI startups and the more potential data customers for Meta since they certainly have large, curated, datasets for sale.
Re:Meta AI releases Code Llama 70B by crowcroft on Monday January 29, 2024 @01:53PM (#39180652)
Meta's end goal is to have better AI than everyone else, in the medium term that means they want to have the best foundational models. How does this help.

1. They become an attractive place for AI researchers to work, and can bring in better staff. 2. They make it less appealing for startups to enter the space and build large foundation models (Meta would prefer 1,000 startups pop up and play around with other people's models, than 1000 startups popping up and trying to build better foundational models). 3. They put cost pressure on AI as a service providers. When LLAMA exists it's harder for companies to make a profit just selling access to models. Along with 2 this further limits the possibility of startups entering the foundational model space, because the path to monetization/breakeven is more difficult.

Essentially this puts Meta, Google, and OpenAI/Microsoft (Anthropic/Amazon as a number four maybe) as the only real players in the cutting edge foundational model space. Worst case scenario they maintain their place in the current tech hegemony as newcomers are blocked from competing.

Re:Meta AI releases Code Llama 70B by siquick on Monday January 29, 2024 @04:14PM (#39182707)
> Essentially this puts Meta, Google, and OpenAI/Microsoft (Anthropic/Amazon as a number four maybe) as the only real players in the cutting edge foundational model space.

Mistral is right up there.

Re:Meta AI releases Code Llama 70B by yodsanklai on Monday January 29, 2024 @04:31PM (#39182963)
Mistral has ~20 employees. I'm sure they have good researchers, but don't they lack the computing and engineering resources the big actors have?
Re:Meta AI releases Code Llama 70B by crowcroft on Monday January 29, 2024 @05:00PM (#39183312)
I'm curious to see how they go, I might have a limited understanding. From what I can tell they do a good job in terms of value and efficiency with 'lighter' models, but I don't put them in the same category as the others in the sense that they aren't producing the massive absolute best in class LLMs.

Hopefully they can prove me wrong though!

Re:Meta AI releases Code Llama 70B by simonw on Monday January 29, 2024 @01:12PM (#39179876)
AI seems like the Next Big Thing. Meta have put themselves at the center of the most exciting growth area in technology by releasing models they have trained.

They've gained an incredible amount of influence and mindshare.

Re:Meta AI releases Code Llama 70B by andy99 on Monday January 29, 2024 @01:20PM (#39180008)
I think a big part of it is just because they have a big AI lab. I don't know the genesis of that, but it has for years been a big contributor, see pytorch, models like SEER, as well as being one of the dominant publishers at big conferences.

Maybe now their leadership wants to push for practicality so they don't end up like Google (also a research powerhouse but failing to convert to popular advances) so they are publicly pushing strong LLMs.

Re:Meta AI releases Code Llama 70B by Philpax on Monday January 29, 2024 @01:15PM (#39179930)
Aside from the "positive" explanations offered in the sibling comments, there's also a "negative" one: other AI companies that try to enter the fray will not be able to compete with Meta's open offerings. After all, why would you pay a company to undertake R&D on building their own models when you can just finetune a Llama?
Re:Meta AI releases Code Llama 70B by two_in_one on Monday January 29, 2024 @07:08PM (#39184570)
Whatever Meta's motivation is they help diversify models suppliers. Which is a good thing not to be locked in. As usual reality is more complicated with many moving part. Free models may undercut small startups. But at the same time they stimulate secondary market of providers and tuners.
Re:Meta AI releases Code Llama 70B by bryan_w on Monday January 29, 2024 @01:12PM (#39179891)
Part of it is that they already had this developed for years (see alt text on uploaded images for example), and they want to ensure that new regulations don't hamper any of their future plans.

It costs them nothing to open it up, so why not. Kinda like all the rest of their GitHub repos.

Re:Meta AI releases Code Llama 70B by Too on Monday January 29, 2024 @02:15PM (#39180993)
Meta still sit on all the juicy user data that they want to use AI on but they don’t know how. They are crowdsourcing development of applications and tooling.

Meta releases model. Joe builds a cool app with it, earns some internet points and if lucky a few hundred bucks. Meta copies app, multiply Joes success story with 1 billion users and earn a few million bucks.

Joe is happy, Meta is happy. Everybody is happy.

Re:Meta AI releases Code Llama 70B by Calvin02 on Monday January 29, 2024 @02:03PM (#39180813)
Controversial take:

Meta sees this as the way to improve their AI offerings faster than others and, eventually, better than others.

Instead of a small group of engineers working on this inside Meta, the Open Source community helps improve it.

They have a history of this with React, PyTorch, hhvm, etc. All these have gotten better as OS projects faster than Meta alone would have been able to do.

Re:Meta AI releases Code Llama 70B by emporas on Monday January 29, 2024 @02:04PM (#39180822)
Yan Le Cunn has talked about Meta's strategy with open source. The general idea, is that the smartest people in the world do not work for you. No company can replicate innovation from open source internally.
Re:Meta AI releases Code Llama 70B by yodsanklai on Monday January 29, 2024 @04:35PM (#39183010)
> The general idea, is that the smartest people in the world do not work for you

Most likely, they work for your competitors. They may not be working to improve your system for free.

> No company can replicate innovation from open source internally.

Lot of innovation does come from companies.

Re:Meta AI releases Code Llama 70B by emporas on Monday January 29, 2024 @06:22PM (#39184151)
>Lot of innovation does come from companies.

Of course, i am not arguing that. But when it comes to software as general as code generation, or text generation, the possible applications are so broad, that a team of A.I. researchers in a company, however talented and productive they are, cannot possibly optimize it for every possible use case.

That's what Yan Le Cunn is referring to, and i agree with him. There are a lot of companies which push deep learning forward, and do not release their code or weights freely.

Re:Meta AI releases Code Llama 70B by flir on Monday January 29, 2024 @02:15PM (#39180989)
Really enjoying how many different answers you got.

(My theory: if there's an AI pot of gold, what megacorp can risk one of the others getting to it first?)

Re:Meta AI releases Code Llama 70B by datadrivenangel on Monday January 29, 2024 @01:24PM (#39180094)
AI puts pressure on search, cutting into google's ad revenue. Meta's properties are less immune to pressure from AI.
Re:Meta AI releases Code Llama 70B by pyinstallwoes on Monday January 29, 2024 @01:10PM (#39179850)
To be crowned the harbinger of AGI.
Re:Meta AI releases Code Llama 70B by apples_oranges on Monday January 29, 2024 @01:10PM (#39179851)
OT: You „don‘t“ or you „don’t fully“ understand? ;)

(I try to train myself to say it right ..)

Re:Meta AI releases Code Llama 70B by theGnuMe on Monday January 29, 2024 @01:50PM (#39180575)
Bill Gurly has a good perspective on it.

Essentially, you mitigate IP claims and reduce vendor dependency.

https://eightify.app/summary/technology-and-software/the-imp...

Re:Meta AI releases Code Llama 70B by blitzar on Tuesday January 30, 2024 @05:51AM (#39188590)
Facebook went all in on the metaverse and turned into Meta; quite rightly, the market looked at what they produced for 10's of billions and decided their company was worthless.

Then Ai sprung to the front pages and any CEO who stood up and said "Ai" was rewarded with a 10x stock price. The unloved stepchild that was the ML team became the A team and the metaverse team have been sent to the naughty step. Facebook/Meta have no actual customer facing use for Ai unlike Microsoft/Google/GitHub but they like a good stonk price rise and so what we see is their stategy to stay in the ai game and relevant.

It turns out it is pretty good for the rest of us (possibly the first time facebook has given something positive to humanity) as we get shinny toys to play with.