News for hackers. Stuff that matters.
Re:Mistral 7B, the most powerful language model for its size... by gfodor on Wednesday September 27, 2023 @05:26PM (#37681562)
Sounds false, this is only an analogy wrapped up in what sounds like an argument. If you think about what you’re actually getting it’s open source.
Re:Mistral 7B, the most powerful language model for its size... by monocasa on Wednesday September 27, 2023 @05:37PM (#37681700)
What do Mistral's engineers work with? Do they have this 14GB pickle file open in their IDEs at 3PM? If not, why is that not the source to this?
Re:Mistral 7B, the most powerful language model for its size... by uoaei on Wednesday September 27, 2023 @10:42PM (#37684544)
It's remarkably telling that no one who's pushing back has any answer to this line of reasoning.

I wonder how they might benefit from retconning the notion of open source to fit the ways they're describing models now. At this point it seems bad faith in more than just a naive way.

Re:Mistral 7B, the most powerful language model for its size... by gfodor on Thursday September 28, 2023 @05:55PM (#37696401)
The reason is because "open source" is a vague enough concept anyone can project what they want onto it. Being able to fully replicate the construction of the model is one lens on it, but being able to modify it to suit your own needs is another.

Given that a released base model allows things like fine tuning, generating your own training sets, and so on, it's not obviously closed source in the way the poster is acting like it is. Releasing the full training data and reproduction pipeline would be "more" open source, but having the raw, non RLHF'ed model is extremely open compared to most alternatives beyond that.

Re:Mistral 7B, the most powerful language model for its size... by monocasa on Thursday September 28, 2023 @08:02PM (#37697630)
It's not as vague as you're making it sound.

Open source would mean what the engineers who work on this build artifact edit. This is clearly not that.

This is 'more open' than other releases, yes. That doesn't mean it reaches the bar of what 'open source' means.

Re:Mistral 7B, the most powerful language model for its size... by gfodor on Friday September 29, 2023 @04:42PM (#37709865)
If a company makes a product that involves a code generator, and publishes the generated code, but not the code generator itself, it’s still open source.
Re:Mistral 7B, the most powerful language model for its size... by uoaei on Friday September 29, 2023 @06:32AM (#37701690)
"Some people are right but if you tilt your head and squint you can almost understand the argument from the others"

I don't know how to respond to these lines of reasoning anymore except to say that words mean things and it's worth keeping definitions consistent for the sake of our common understanding.

Re:Mistral 7B, the most powerful language model for its size... by gfodor on Friday September 29, 2023 @04:45PM (#37709883)
Machine learning models force us to reconsider the definitions or introduce new ones. A published model has characteristics of open source and closed source software, given the extent to which it can be fundamentally modified, inspected, and used to generate derivative works in a way executable binaries cannot.