News for hackers. Stuff that matters.
Re:LLaMA: A foundational, 65B-parameter large language model by recuter on Friday February 24, 2023 @11:43AM (#34926487)
I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react.

In the case of OpenAI it is a "nudge humanity into a more wholesome direction" because a lot of them went on an acid fueled toxic affective altruism bender. And in this case it is "release all the things while Zuck is still obsessed with VR --for science!".

I like this other group better. But it is disturbing that it can stop at any moment. Probably why they are doing it, while they still can.

Re:LLaMA: A foundational, 65B-parameter large language model by titzer on Friday February 24, 2023 @12:15PM (#34926909)
They almost certainly spent at least a few million dollars on this research project. Hard to say when the decision was made to open source this (from the outset or after it started showing results), but the decision was conscious and calculated. Nothing this high-profile is going to escape the strategic decision making processes of upper management.
Re:LLaMA: A foundational, 65B-parameter large language model by fwlr on Friday February 24, 2023 @11:51AM (#34926588)
The researchers and engineers and other assorted dorks who built it weren’t thinking about moats, for sure, I agree with you there. But I guarantee you that the metaphor of medieval warfare was on the minds of the executives and legal team deciding whether to let the team release their work in this way.
Re:LLaMA: A foundational, 65B-parameter large language model by visarga on Friday February 24, 2023 @12:37PM (#34927187)
> It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given

Probably not, Zuck is announcing it.

Re:LLaMA: A foundational, 65B-parameter large language model by recuter on Friday February 24, 2023 @12:56PM (#34927486)

  Today we're releasing a new state-of-the-art AI large language model called LLaMA designed to help researchers advance their work. LLMs have shown a lot of promise in generating text, having conversations, summarizing written material, and more complicated tasks like solving math theorems or predicting protein structures. Meta is committed to this open model of research and we'll make our new model available to the AI research community.
I don't know what that means or if he even wrote/read it tbh. I hope it literally just means Meta is actually committed to this open model of research (for now).

Maybe he is being a Machiavellian moat filler, I stand corrected. I think/hope that they don't really have a plan to counter OpenAI yet because I am afraid this attitude won't last once they do and this stuff has recently started moving quickly.

Re:LLaMA: A foundational, 65B-parameter large language model by ahati on Friday February 24, 2023 @02:34PM (#34928920)
It is to please the shareholders. Now shareholder can know that Meta can compete with GPT3.
Re:LLaMA: A foundational, 65B-parameter large language model by ttul on Friday February 24, 2023 @05:20PM (#34930953)
100%. This kind of announcement is for the street. Investors need to be reassured that Meta is keeping up with the cool AI stuff over at Google and Microsoft. A release like this will cause a flurry of downstream news coverage. I’m sure they are hoping that the model will be picked up by researchers doing interesting things who might further generate good coverage.

And, yes, it fills the moat.

Re:LLaMA: A foundational, 65B-parameter large language model by mliker on Friday February 24, 2023 @12:25PM (#34927015)
this is a pretty biased and uninformed opinion. Pretty condescending to call it a "bunch of dorks...running around unsupervised."
Re:LLaMA: A foundational, 65B-parameter large language model by recuter on Friday February 24, 2023 @12:29PM (#34927071)
I don't know what my bias is supposed to be but I called them dorks affectionately for one. The other replies are literally arguing that they are very extremely supervised where as I am speculating they are just eager to share their work for the right reasons and the eye of Sauron has yet to turn upon them.

Inside knowledge I never claimed. Anything else I can help you with today? :)

Re:LLaMA: A foundational, 65B-parameter large language model by rcpt on Friday February 24, 2023 @01:59PM (#34928433)
> I called them dorks affectionately

Never in my life have I seen that word used affectionately

Re:LLaMA: A foundational, 65B-parameter large language model by vidarh on Friday February 24, 2023 @02:30PM (#34928881)
I have seen that many times, as well as seeing people use it about themselves.
Re:LLaMA: A foundational, 65B-parameter large language model by sebastiennight on Friday February 24, 2023 @05:18PM (#34930943)
I think Noah Kagan's blog is literally called "OK Dork", and I always assumed the title was self-deprecating (in a fun/positive way) rather than negative.
Re:LLaMA: A foundational, 65B-parameter large language model by drusepth on Friday February 24, 2023 @08:49PM (#34932919)
Geek, nerd, and dork can all be used positively/affectionately. I know I've called myself and friends all of the above on many occasions.
Re:LLaMA: A foundational, 65B-parameter large language model by alibero on Saturday February 25, 2023 @12:30AM (#34934178)
As one of the unsupervised dorks working on LLMs at Meta (not one of the authors here) I took it in a positive way :)
Re:LLaMA: A foundational, 65B-parameter large language model by recuter on Saturday February 25, 2023 @02:31AM (#34934697)
Thank you for your service meatbag <3
Re:LLaMA: A foundational, 65B-parameter large language model by tantalor on Friday February 24, 2023 @01:37PM (#34928112)
Check out https://news.ycombinator.com/newsguidelines.html

Not sure how calling out comment as "biased", "uninformed" or "condescending" is helping.

Re:LLaMA: A foundational, 65B-parameter large language model by vineyardmike on Friday February 24, 2023 @01:08PM (#34927660)
I think there is definitely a component of competition here.

Facebook has no real way to monetize this (eg they won’t release an api a la OpenAI and they don’t own a search engine). Since they can’t monetize it… why not provide a bit of kindling to lower the barrier for everyone else to compete with your competitors. This strategy is called “commodize your complement”.

If Facebook makes it easier to develop a google alternative, especially by doing something that doesn’t hurt you, you just weaken a competitor. See Facebook releasing datasets for mapping. Think of the panic ChatGPT caused Google. It only cost a few Million to train but it’s probably costing Google more than that already.

Re:LLaMA: A foundational, 65B-parameter large language model by naasking on Friday February 24, 2023 @01:55PM (#34928368)
Facebook maybe can't make money from it, but they arguably could save money from it, for instance in automating some fact checking and moderation activities that they currently spend quite a bit of money on.
Re:LLaMA: A foundational, 65B-parameter large language model by sclarisse on Friday February 24, 2023 @10:39PM (#34933620)
If you’re going to ask an AI to do fact checking with today’s technology, I would urge you to start by asking the AI a simple test question: Which is heavier? A pound of feathers or two pounds of lead?
Re:LLaMA: A foundational, 65B-parameter large language model by elbear on Saturday February 25, 2023 @02:21AM (#34934645)
Do you have an explanation for why it fails on this question?
Re:LLaMA: A foundational, 65B-parameter large language model by panarky on Saturday February 25, 2023 @03:26AM (#34934930)
Why assume it fails?

ChatGPT: "One pound of feathers and two pounds of lead weigh the same, which is one pound or 16 ounces. The difference is in their volume, where a pound of feathers takes up more space than two pounds of lead. This is because the density of feathers is much less than that of lead, so even though the weight is the same, the amount of space they occupy is quite different."

Re:LLaMA: A foundational, 65B-parameter large language model by espadrine on Saturday February 25, 2023 @05:04AM (#34935428)
Your question makes it sound like you believe it succeeds, but the answer you pasted is incorrect.

As for why it fails, it is likely a bias arising from the question being way more commonly asked in the corpus with equal mass than with distinct mass, increasing attention weights towards an answer expressing equality.

I believe current LLMs lack some common sense at an architectural level. They learn both specialized facts and general deduction in the same weights class: in my mind, they should separate their world model from their instance model.

Re:LLaMA: A foundational, 65B-parameter large language model by selfhoster11 on Saturday February 25, 2023 @07:55AM (#34936299)
I was going to provide evidence that this works correctly, but I was surprised to find that it gets severely tripped up on this particular question, even when told to “show working step by step”.
Re:LLaMA: A foundational, 65B-parameter large language model by webmaven on Friday February 24, 2023 @02:49PM (#34929150)
> Facebook has no real way to monetize this (eg they won’t release an api a la OpenAI and they don’t own a search engine). Since they can’t monetize it… why not provide a bit of kindling to lower the barrier for everyone else to compete with your competitors. This strategy is called “commodize your complement”.

Your analysis is good, but that isn't what "commoditize your complement" means.

Strictly speaking, a search engine isn't a complement for FBs revenue streams. Relatively little of FBs revenue can be attributed to search engine traffic leading to FBs walled garden where they can show the user ads.

Complements are generally a required product or service that enables your core, but that isn't revenue generating for you. Examples for FB are data centers (so they participate in the Open Compute Project[0]), and mobile operating systems (which Google already has made a commodity, for their own reasons, with Android).

What FB is doing here is commoditizing their competitors' core offering (or rather, a rather promising future one). That's just the tactic though, there are several strategies this can enable, from undermining the barriers to entry into their competitors' search markets, to actually fragmenting that market by encouraging a diversity of specialized chat interfaces over one big chat model. You can see hints of both in this announcement.

Final note: FB is also protecting itself from relying on a competitor as a supplier should chat become a preferred user interface for the content on social networks, which it hasn't, but if it ever did this would count as "commoditizing their complement", though I would actually expect FB to switch to a mostly proprietary approach in that circumstance (so not much openness on having LLMs operate on social graphs and the like), just keeping the foundation they rely on, and which undermines and prevents gatekeeping by their advertising competitors, open.

[0] https://www.opencompute.org/

Re:LLaMA: A foundational, 65B-parameter large language model by invig on Friday February 24, 2023 @04:54PM (#34930656)
I don't think that's the best interpretation of complement. Think of complement as "is your users experience benefited by the existence of the other thing". I think that Search qualifies as something that enhances the Facebook users experience (e.g. my cousin mentioned thing that they like doing today, how do I find more about thing?)
Re:LLaMA: A foundational, 65B-parameter large language model by webmaven on Thursday March 02, 2023 @04:59PM (#35002661)
Given FBs optimization of the feed for dopamine hits, I don't think search across posts is a high priority (yes, it is table stakes, but whatever they currently use seems to be commoditized enough for their purposes). Group and page recommendations and discovery is another matter, perhaps an LLM may help make that more engaging. It might also be useful for helping users (and brands) moderate their groups and so on. These are also complements, but they aren't external ones.