News for hackers. Stuff that matters.
Re:Open source LLM with 32k Context Length by supermatt on Thursday August 24, 2023 @03:03AM (#37245890)
It seems this is built on LLAMA. Did meta change the license to make it open source now? It still seems to be showing otherwise in the repo.

Edit: No mention of it being open source in the linked article. Maybe the title here is just wrong? @dang

Re:Open source LLM with 32k Context Length by sillysaurusx on Thursday August 24, 2023 @03:11AM (#37245941)
It’s not possible to have a license over an ML model trained on other peoples’ works, since such models are uncopyrightable. They’re more like a phone book; a collection of facts trained by an entirely un-creative process. https://news.ycombinator.com/item?id=36691050

This hasn’t been proven in court, but it seems the most likely outcome.

Re:Open source LLM with 32k Context Length by supermatt on Thursday August 24, 2023 @03:25AM (#37246029)
And yet meta are specifying a license, implying they do hold the copyright.
Re:Open source LLM with 32k Context Length by sillysaurusx on Thursday August 24, 2023 @03:29AM (#37246047)
They’re mistaken.

It’ll take awhile for this mistake to be reconciled in court though.

Re:Open source LLM with 32k Context Length by nemoniac on Thursday August 24, 2023 @08:26AM (#37247987)
Not saying that this applies to LLMs but if you describe them as "a collection of facts [collected and] trained by an entirely un-creative process" then it begins to sound like one could argue for Database Right.

https://en.wikipedia.org/wiki/Database_right

Re:Open source LLM with 32k Context Length by inciampati on Thursday August 24, 2023 @09:49AM (#37248755)
Keep working on it! A good court case and a landmark decision about this could change the landscape for the market for these models.
Re:Open source LLM with 32k Context Length by ImprobableTruth on Thursday August 24, 2023 @03:09AM (#37245933)
Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.
Re:Open source LLM with 32k Context Length by supermatt on Thursday August 24, 2023 @03:17AM (#37245979)
If the JSON license isn't considered open, due to requiring that "The Software shall be used for Good, not Evil.", then I don't see how tacking an additional financial threshold onto it makes it more open. I don't think meta even released the training dataset, so you cant even replicate it (should you have the funds to do so).

There are other LLMs that don't have such restrictions, and publish their training data.

Re:Open source LLM with 32k Context Length by satvikpendem on Thursday August 24, 2023 @04:29AM (#37246373)
There is no open source-ish. It either protects the fundamental freedoms or it...doesn't.
Re:Open source LLM with 32k Context Length by pastage on Thursday August 24, 2023 @06:15AM (#37247098)
Sure there is, BSD is bad because xyz, GPL is bad because zyx. That said Llama restrictions are rather harsh and you are not allowed to improve other models with it. So no freedom there just some ok beer.
Re:Open source LLM with 32k Context Length by jmiskovic on Thursday August 24, 2023 @06:16AM (#37247107)
Open source is both a colloquial term for available/modifiable/distributable code (like Llama2), and a strict OSI-approved list of licenses. I'd say opensource-ish is a great fit here.
Re:Open source LLM with 32k Context Length by keyle on Thursday August 24, 2023 @05:04AM (#37246597)
"open source-ish" sounds like the perfect way to massively profitable future litigations.

Also "don't do naughty things", is there a chart for that? How is that defined, is it part of the non-existing license?

Re:Open source LLM with 32k Context Length by yreg on Thursday August 24, 2023 @08:33AM (#37248037)
> How is that defined, is it part of the non-existing license?

What do you mean? https://github.com/facebookresearch/llama/blob/main/LICENSE

Re:Open source LLM with 32k Context Length by jmiskovic on Thursday August 24, 2023 @06:05AM (#37247021)
Nope. It's limited by 700m monthly-active users at the time Llama2 was released, a weird catch clause for a handful of Meta competitor companies. The license doesn't satisfy OSS requirements, but it is quite reasonable.

https://ai.meta.com/llama/license/ https://ai.meta.com/llama/use-policy/

Re:Open source LLM with 32k Context Length by Havoc on Thursday August 24, 2023 @05:45AM (#37246892)
From memory llama 2 license does allow tuned models with suitable credit & license inclusion. The restricted using it to train other models though (a bit like people use gpt4 to generate question/answer pairs to train their models)