It's very nice that they released their build artifacts. It's great that you can take that and make small modifications to it. That doesn't make it open source.
> For most of these LLMs (not sure about this one in particular yet) the energy cost in particular of recreation is more than most individuals earn in a lifetime, and the enormous data volume is such that the only people who seriously need this should be copyright lawyers and they should be asking for it to be delivered by station wagon.
All of that just sounds like reasons why it's not practical to open source it, not reasons why this release was open source.
I could either point out that JPEG is an algorithm, or ask if you can recreate a sunset.
> All of that just sounds like reasons why it's not practical to open source it
No, they're reasons why the stuff you want doesn't matter.
If you can actually afford to create a model of your own, you don't need to ask: the entire internet is right there. Some of it even has explicitly friendly licensing terms.
An LLM with a friendly license is something you can freely integrate into other things which need friendly licensing. That's valuable all by itself.
There's been all sorts of closed source libraries that you can freely integrate for whatever reason. They're not open source either.
Once again, this pickle file is not what's sitting in Mistral's engineer's editors as they go about their day.
Please be clear, we keep asking. What are you asking for? Datasets? Training algo? What?
Comparing it to software artifacts isn't a good comparison when any program with open source code (visible or free to use) is equivalent to what's being given. You have everything you need to use, edit, and fuck around with. You don't have the exact scheme, but I'll put it this way, if you gave me the hardware I could produce a LLM of high quality from scratch using their architecture.
[0] https://github.com/mistralai/mistral-src/blob/main/mistral/m...
It's not what Mistral's engineers edit to create this release. Just like an ELF file is necessarily contains the code flow graph, in a way extractable by experts, but isn't open source because... it's not source.
The open source family of licenses are about freedom. If I’m not given the tools to recreate a model, then I’m not afforded the freedoms normally associated with these open licenses. Really there’s little difference between Apache and CC-BY here.
You've gotta give me the stuff you used to make it, the stuff you'd want to have if you wanted to recreate a slightly different version of the photo ("in the preferred form for making modifications", as the GPL says). If you just snapped a photo of whatever you saw with whatever camera was in your pocket, then there's nothing else to publish. But if you figured out a timetable of when you should stand where with what kind of lens, then making your photo open-source would mean publishing that timetable.
> These models are perfectly valid things in their own right; the can be fine-tuned or used as parts of other things.
If the original creator can edit them, and you can't, then that's not open-source; fine-tuning is a help but someone who can only fine-tune is still a second-class user compared to the original developer. The whole point of open source is to put you on an equal footing with the original developer (in particular, to make sure that you can fix bugs by yourself and are never stuck waiting for them to release an update that you need).
The MIT license doesn't distinguish between binaries and source, but the GPL does. The GPL defines source code as:
> The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work.
What is the preferred form of an LLM for making modifications to it?
Phorographs are not source code, are not computer code at all, and do not have a close analog to source code. Calling them “open source” is, at best, a poor and distant metaphor in any case, and so they aren't a useful model for discussing what open source means for software.
There's a very reasonable argument that model weights are an IL or object-code like artifact with training data and training source code together as the source code.
That doesn't change that the MIT license is an open source license, but when what you release under that license isn’t the whole source aoplicable to the model, but just inference and maybe training code but not the data needed to produce the weights, and final weights, then it is fair to question whether as a whole the model is open source.
Open source as a concept doesn't really apply to quite a lot of things. Your MIT-licensed photograph is "not open source" in the same way that `{} * {}` is "not a number" (it technically isn't, but that's not quite what NaN is supposed to mean).
So if I take a photo of a pretty sunset, release it under MIT license, you'd say it's "not open source" unless I give you the sun and the atmosphere themselves?
These models are perfectly valid things in their own right; the can be fine-tuned or used as parts of other things.
For most of these LLMs (not sure about this one in particular yet) the energy cost in particular of recreation is more than most individuals earn in a lifetime, and the enormous data volume is such that the only people who seriously need this should be copyright lawyers and they should be asking for it to be delivered by station wagon.