News for hackers. Stuff that matters.
Re:Mistral 7B, the most powerful language model for its size... by monocasa on Wednesday September 27, 2023 @09:57PM (#37684246)
So the engineers that built this edited this release directly? Or did they write code that built this?
Re:Mistral 7B, the most powerful language model for its size... by jncfhnb on Wednesday September 27, 2023 @10:15PM (#37684380)
Every engineer uses tools to create their products. Those products that are open source do not include the tools used to create them (usually).
Re:Mistral 7B, the most powerful language model for its size... by monocasa on Wednesday September 27, 2023 @10:20PM (#37684417)
"Open source" normally contains what the engineers working on it edit to submit a job to build into the build artifacts. This does not contain that, but instead the result of the build process.
Re:Mistral 7B, the most powerful language model for its size... by jncfhnb on Thursday September 28, 2023 @09:18AM (#37689196)
That’s because you’re failing to differentiate a build process meaning compiling your stuff to an end result, vs. fine tuning an algorithm to refine code.
Re:Mistral 7B, the most powerful language model for its size... by monocasa on Thursday September 28, 2023 @08:08PM (#37697681)
Explicitly.

I regard the distinction as meaningless and haven't heard a good reason as why I should reconsider when the process behind creating the weights is so integral to the overall engineering process here.

Re:Mistral 7B, the most powerful language model for its size... by jncfhnb on Thursday September 28, 2023 @08:43PM (#37697925)
Well fortunately that’s just your opinion and it is neither popular nor relevant.

To quote stallman

> The "source code" for a work means the preferred form of the work for making modifications to it

That’s what this is. This is not a work that produces weights. So the code related to making weights would not be the source code of this project. It would be the source code of the tools used to make this project.

Re:Mistral 7B, the most powerful language model for its size... by monocasa on Thursday September 28, 2023 @08:57PM (#37698014)
That quote is specifically designed to back up my position, hence why I've quoted it else where in this thread.

This is evidenced by the fact that if Mistral's engineers had to make modifications to this model, they would use other code to do so. This model is not the "preferred form of the work for making modifications to it", but simply the best we have in a world that isn't used open sourcing these models.

Re:Mistral 7B, the most powerful language model for its size... by jncfhnb on Thursday September 28, 2023 @11:01PM (#37698845)
> This is evidenced by the fact that if Mistral's engineers had to make modifications to this model, they would use other code to do so.

It’s a PyTorch model. There is no secret internal code required to interact with it. You can write whatever you want. Unless you’re implying that in order to call this model open source they need to get all of PyTorch on the same license. You can write whatever you want to edit the weights.

Open source does not require babysit walking people through the basic competency steps of understanding how the tools of this domain work.

This model is 100% the preferred form of work for making modifications to it as evidenced by the large community oriented around sharing and adapting models from exactly this format.

And no, I’m calling you for changing your argument again. You were saying dumb shit about how they needed to open source all the data needed to train this thing from scratch, which is another wildly different argument.