News for hackers. Stuff that matters.
Re:Qwen2 LLM Released by diggan on Thursday June 06, 2024 @01:17PM (#40599958)
The model seems to be Apache 2.0 yet the training data remains private, so even if you had the resources, knowledge and ability to train it yourself, you wouldn't be able to.

So no, Qwen 2 isn't open source, but they happen to release the models publicly. Guess "pseudo-open source" might make sense as a label.

I agree, I'm not a super fan of people/organizations using "open source" as a marketing term which seems popular in the ML field right now.

Re:Qwen2 LLM Released by okaram on Thursday June 06, 2024 @02:11PM (#40600559)
I think we need to agree on terminology, but to me, this seems clear.

The model is open-source (or open-content, if you prefer). The input data isn't.

Re:Qwen2 LLM Released by loudmax on Thursday June 06, 2024 @02:31PM (#40600776)
I like the term open-weights.

The "source", analogous to the source code for a program, should include the training data. In this case that isn't open. The resulting weights are open, insofar as they can be redistributed and fine-tuned and so on.

Re:Qwen2 LLM Released by Zambyte on Thursday June 06, 2024 @02:23PM (#40600687)
The input data is the source (literally "where it comes from"). If the source is not open, it is not open source.

The model is open weight, despite an OSI approved license sitting in the same directory as the binary blob.

Re:Qwen2 LLM Released by anon373839 on Thursday June 06, 2024 @07:47PM (#40603782)
I like open-everything as much as the next person, but I don’t really agree with this position. The source code to run the model is open, as are the weights of the thing you’re running. I think it’s fair to say that they have open-sourced an LLM inference system. This isn’t traditional software, and it’s not clear how the term “open source” should be defined.

To stretch the analogy a different way, it could have been argued that PyTorch isn’t “open source” because the repo doesn’t include the private notes, sketches and communications of the team that developed it. How could someone reproduce the source code for themselves without access to the inputs that went into designing it?

Of course, we don’t define “open source” in that way for source code. But we could have.

Re:Qwen2 LLM Released by Zambyte on Thursday June 06, 2024 @09:19PM (#40604368)
> The source code to run the model is open

You can't build the model from source with code. That's because the input data is an essential source of the model.

> I think it’s fair to say that they have open-sourced an LLM inference system.

Maybe they have. That's separate from the model though, and a lot of people use different, more standardized inference systems (Ollama, vLLM, etc).

> it could have been argued that PyTorch isn’t “open source” because the repo doesn’t include the private notes, sketches and communications of the team that developed it.

Those aren't inputs used to build a runnable package of PyTorch. The source of some binary is the human readable and editable input used to produce the binary. Notes and communications are human readable input to the already human readable code; it's therefore not a source for binaries build from the code.

LLM Weights are not human readable nor human editable. They are machine readable (through inferencing) and machine editable (through fine tuning). If that counts as open source, then so is any binary executable since patchelf and co exist.

Re:Qwen2 LLM Released by dTal on Thursday June 06, 2024 @07:48PM (#40603789)
I advocate applying the original term for such things, "freeware".

No need to needlessly complicate things.