LLaMA2 isn't “Open Source” – and why it doesn't matter

Re:LLaMA2 isn't “Open Source” – and why it doesn't matter by concinds on Friday July 21, 2023 @04:01PM (#36818902)

Fundamentally untrue, and disheartening that it's the top comment.

You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2

And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE

[ Parent | on HN ]

Re:LLaMA2 isn't “Open Source” – and why it doesn't matter by rgoldste on Friday July 21, 2023 @04:06PM (#36818968)

The truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.

[ Parent | on HN ]

Re:LLaMA2 isn't “Open Source” – and why it doesn't matter by int_19h on Friday July 21, 2023 @08:27PM (#36821891)

Some of the best LLaMA finetunes today are trained on GPT-4 output.

Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.

[ Parent | on HN ]