News for hackers. Stuff that matters.
Re:Mistral 7B, the most powerful language model for its size... by Dowwie on Wednesday September 27, 2023 @07:28PM (#37682926)
You want to see the source code of the transformer that created the compiled llm model, the dictionary corresponding to token values, etc. Instead, you got the weights and binary.

I don't know what's going on in this discussion thread, as to whether you are being gaslit or engaging with people who shouldn't comment.

Re:Mistral 7B, the most powerful language model for its size... by uoaei on Wednesday September 27, 2023 @10:40PM (#37684537)
Right, my interpretation (and I think monocasa's too) is that the training and packaging pipeline is not published therefore the code used to build the model is not open, therefore there is no source available to rebuild the model as Mistral's engineers have done.

In essence the output of the training pipeline is "data" (weights), not code. The way those data are read to make things work is the "model". In the (bad) analogy of the JPEG, the data is the coefficients and the model is the encoding/decoding scheme.

I'm impressed by the patience monocasa has in the face of so many (seemingly intentionally) obtuse responses.

Re:Mistral 7B, the most powerful language model for its size... by artursapek on Wednesday September 27, 2023 @10:59PM (#37684664)
The JPEG analogy makes no sense. A more correct analogy is: they released compiled binaries, not the source code.

The difference is though, here compiling from source is prohibitively expensive for the average person. And the source is massive.

Re:Mistral 7B, the most powerful language model for its size... by uoaei on Thursday September 28, 2023 @03:18AM (#37686395)
My analogy makes perfect sense.

Also, being able to reverse-engineer the precise build process is an orthogonal concern to having the source code available.